Test Automation
The essentials to start and optimize automated testing: concepts, pros and cons, steps, what to automate, design patterns, metrics and tools.
Based on ISTQB CTAL-TAE v2.0
Test automation is the use of tools and frameworks to control, configure, run, validate and report tests without human intervention. It increases speed, repeatability and test coverage, covering software with or without a UI, mobile apps and network protocols.
Automation does NOT replace manual testing: it complements it, especially in regression, smoke and repetitive tests.
Key concepts
Flaky test
An unstable test that sometimes passes and sometimes fails without code changes (timing, data, external dependencies).
SUT (System Under Test)
System under test: an application, module or component.
TAA (Test Automation Architecture)
High-level technical design: components, layers, interfaces and integration.
TAF (Test Automation Framework)
Set of tools, libraries, standards and best practices supporting automation.
CI/CD
Continuous integration and delivery: integrate and deliver software with automated validation.
Parallelism
Simultaneous test execution to reduce time and optimize feedback.
Advantages
- Run tests that cannot be done manually and more complex ones.
- Run tests faster and more of them per build.
- Minimize the risk of failures from human intervention.
- Use test resources more effectively and efficiently.
- Provide faster feedback and improve system reliability.
Disadvantages
- Initial investment to set up the solution.
- Time to develop and maintain the solution.
- Requires clear automation goals to ensure success.
- Additional costs: new professionals, resources and training.
- Test rigidity, less adaptability to SUT changes.
Steps to implement
- Analyze feasibility and goals.
- Identify what to automate, define criteria and metrics.
- Design the TAA (architecture) and TAF (framework).
- Develop initial scripts.
- Integrate with CI/CD.
- Execution and report generation.
- Continuous maintenance and optimization.
What to automate and what not?
| Do automate | Don’t automate |
|---|---|
| Regression and repetitive tests. | Exploratory and usability tests. |
| Critical, high-risk scenarios. | Frequently changing tests. |
| Load and performance tests. | Complex visual checks. |
| Cases with large data volumes. | One-off or single-run tests. |
| Smoke and sanity tests; stable business rules. | Features under active development. |
Best practices
- Modular, reusable design (TAA and TAF), following patterns like Page Object Model.
- Strategic selection: prioritize stable, repetitive, high-value cases.
- Document, version and maintain code; detect and remove flaky tests.
- Early, continuous integration (Shift-Left + CI/CD), running on every commit.
- Manage controlled test data separate from scripts (Data-Driven).
- Clear, actionable reports with key metrics, and collaboration with Dev, DevOps and manual QA.
Design patterns
Facade
Hides internal implementation and exposes only what is needed. E.g.: LoginFacade.login(user, pass).
Singleton
Guarantees a single instance with one access point. E.g.: a single SUT driver instance.
Page Object Model (POM)
One class per page or component centralizing locators and actions. Eases maintenance.
Flow model
An extension of POM: a layer storing reusable business flows (e.g. Login + Transfer).
Metrics and KPIs
- Test and code coverage (% automated / % covered).
- Execution success rate (% passing without errors).
- Flaky test rate and false positive/negative rate.
- Suite execution time and maintenance effort.
- Return on investment (ROI): benefits versus total costs.
Tools and frameworks
| Category | Common languages |
|---|---|
| UI | JavaScript, TypeScript, Python, Java, C# |
| API | JavaScript, TypeScript, Java, Gherkin-based DSL |
| Mobile | Java, Kotlin, Swift, Objective-C, JavaScript |
| CI/CD | Pipelines in Groovy or YAML |
Final tips
Trust your judgment as a Quality Engineer. Communicate when blocked, adapt to new tools, and use AI to your advantage while always reviewing code critically. Keep the team aligned on context, practices and protocols.
Save or share this content
Download it as PDF or Markdown to save or share it.