TestNG Testing Skill
Overview
This skill provides production-grade TestNG test generation patterns for Java projects. It covers groups, data providers, parallel execution, XML suite configuration, listeners, soft assertions, and lifecycle annotations. Use the core patterns below for everyday work; load the deep playbook reference when the task requires advanced configuration such as thread-safe drivers, factories, CI/CD integration, or reporting frameworks.
When to Use
Use this skill when the user needs to generate or configure TestNG tests in Java. Trigger keywords and phrases:
- "TestNG"
- "@DataProvider"
- "testng.xml"
- "TestNG suite"
- "parallel tests Java"
- "groups" (in a Java testing context)
- "@Listeners", "ITestListener"
- "SoftAssert"
- "dependsOnMethods"
- "Surefire TestNG"
Do not use this skill for JUnit-only projects. If the user mentions JUnit without TestNG, defer to a JUnit skill.
Prerequisites
- Java 8+ installed and on
PATH. Verify:java -version - Maven 3.6+ installed and on
PATH. Verify:mvn -version - TestNG dependency in
pom.xml:<dependency> <groupId>org.testng</groupId> <artifactId>testng</artifactId> <version>7.10.2</version> <scope>test</scope> </dependency> - Maven Surefire Plugin configured to use TestNG and point at the suite XML:
<plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-surefire-plugin</artifactId> <version>3.2.5</version> <configuration> <suiteXmlFiles> <suiteXmlFile>testng.xml</suiteXmlFile> </suiteXmlFiles> </configuration> </plugin> - On Windows (PowerShell), ensure line endings in
testng.xmlare LF or CRLF consistently; mixed endings can cause Surefire parsing errors.
Procedure
1. Identify the test scenario
Determine which TestNG features the user needs:
| Need | Feature |
|---|---|
| Run subsets of tests | Groups (@Test(groups = "smoke")) |
| Feed multiple data sets | @DataProvider |
| Run tests concurrently | Parallel execution in testng.xml |
| Configure multi-env suites | testng.xml suite XML |
| Capture failures, retries, screenshots | @Listeners + ITestListener |
| Collect multiple assertion failures | SoftAssert |
| Cross-browser matrix | @Factory (see playbook §5) |
2. Write the test class
Use the lifecycle annotation order as a mental model:
@BeforeSuite → @BeforeTest → @BeforeClass → @BeforeMethod → @Test → @AfterMethod → @AfterClass → @AfterTest → @AfterSuite
Basic test with groups:
import org.testng.annotations.*;
import org.testng.Assert;
public class LoginTest {
@BeforeMethod
public void setUp() { /* setup */ }
@Test(groups = "smoke")
public void testLoginSuccess() {
Assert.assertTrue(loginService.login("user@test.com", "password123"));
}
@Test(groups = "regression", dependsOnMethods = "testLoginSuccess")
public void testAccessDashboard() {
Assert.assertNotNull(dashboard.getContent());
}
@Test(expectedExceptions = AuthenticationException.class)
public void testLoginInvalidPassword() {
loginService.login("user@test.com", "wrong");
}
@AfterMethod
public void tearDown() { /* cleanup */ }
}
3. Add data providers when test data is reusable
@DataProvider(name = "loginData")
public Object[][] loginData() {
return new Object[][] {
{"admin@test.com", "admin123", true},
{"user@test.com", "password", true},
{"invalid@test.com", "wrong", false},
};
}
@Test(dataProvider = "loginData")
public void testLogin(String email, String password, boolean expected) {
Assert.assertEquals(loginService.login(email, password), expected);
}
For advanced data providers (Excel, JSON, CSV, parallel, cross-class), load references/playbook.md §4.
4. Create the testng.xml suite file
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Regression" parallel="tests" thread-count="5">
<test name="Smoke">
<groups><run><include name="smoke"/></run></groups>
<classes><class name="tests.LoginTest"/></classes>
</test>
<test name="Full">
<groups><run><include name="regression"/><exclude name="flaky"/></run></groups>
<packages><package name="tests.*"/></packages>
</test>
</suite>
5. Configure parallel execution
Choose the granularity:
<suite parallel="methods" thread-count="5"> <!-- Method level -->
<suite parallel="classes" thread-count="5"> <!-- Class level -->
<suite parallel="tests" thread-count="5"> <!-- Test level -->
For thread-safe driver patterns (ThreadLocal, ConfigReader), load references/playbook.md §3.
6. Use soft assertions for multi-check scenarios
SoftAssert soft = new SoftAssert();
soft.assertEquals(user.getName(), "Alice");
soft.assertEquals(user.getAge(), 25);
soft.assertTrue(user.isActive());
soft.assertAll(); // Reports all failures at once
7. Add listeners for production-grade reporting
public class TestListener implements ITestListener {
@Override public void onTestFailure(ITestResult result) {
System.out.println("Failed: " + result.getName());
// Take screenshot, log, etc.
}
}
@Listeners(TestListener.class)
public class LoginTest { /* ... */ }
For production listener suites (retry analyzers, screenshot capture, timing), load references/playbook.md §6.
8. Run the tests
| Task | Command (PowerShell) |
|---|---|
| Run suite | mvn test -DsuiteXmlFile=testng.xml |
| Run group | mvn test -Dgroups=smoke |
| Run single class | mvn test -Dtest=LoginTest |
| Run single method | mvn test -Dtest=LoginTest#testLoginSuccess |
9. Check reports
Default TestNG HTML report:
test-output\index.html
Open in a browser:
Start-Process test-output\index.html
For Allure or ExtentReports integration, load references/playbook.md §10.
Pitfalls
Hard rules — do not violate
- Never use
dependsOnMethodseverywhere. Cascading failures make debugging impossible. Prefer independent tests; usedependsOnGroupssparingly for true ordering dependencies. - Never omit groups. Without
@Test(groups = "...")you cannot run subsets via-Dgroups. Always assign at least one group. - Never hard-code test data inside test methods when a data provider is appropriate. Use
@DataProviderfor reusable, maintainable data sets. - Never rely on
priorityordering as a substitute for independent tests. Priority-based ordering is fragile and breaks when tests are added or removed. - Never assume thread safety in parallel mode. When
parallel="methods"or higher, shared state (e.g., a staticWebDriver) will cause race conditions. UseThreadLocalper-thread drivers. - Never mix line endings in
testng.xmlon Windows. Ensure consistent LF or CRLF; mixed endings can cause Surefire XML parsing failures.
Anti-patterns quick reference
| Bad | Good | Why |
|---|---|---|
dependsOnMethods everywhere |
Independent tests | Cascading failures |
| No groups | @Test(groups = "smoke") |
Can't run subsets |
| Hard-coded test data | @DataProvider |
Reusable |
| Priority ordering | Independent tests | Fragile |
| Shared static driver in parallel | ThreadLocal<WebDriver> |
Race conditions |
Assert after every check in long method |
SoftAssert.assertAll() at end |
Only first failure reported |
Common debugging problems
For the full 12-problem debugging quick-reference, load references/playbook.md §12.
Verification
After generating or modifying TestNG tests, verify with these checkable steps:
Compile the project:
mvn compile test-compileExpected:
BUILD SUCCESSRun the suite and confirm tests execute:
mvn test -DsuiteXmlFile=testng.xmlExpected:
BUILD SUCCESSwithTests run: N, Failures: 0, Errors: 0, Skipped: 0Run a specific group to confirm group filtering works:
mvn test -Dgroups=smokeExpected: only tests annotated
@Test(groups = "smoke")execute.Verify the HTML report was generated:
Test-Path test-output\index.htmlExpected:
TrueVerify parallel execution by checking thread count in logs — TestNG logs thread pool usage when
parallelis set. Confirmthread-countmatches the suite XML value.Verify data provider invocation count — a test using
@DataProviderwith 3 data rows should showTests run: 3for that single method.
References
The following reference file contains deep patterns beyond the core skill:
| File | When to Load |
|---|---|
references/playbook.md |
When the task requires advanced configuration: Maven/Surefire setup, multi-env suite XML, ThreadLocal drivers, Excel/JSON/CSV data providers, @Factory cross-browser matrix, production listeners (retry, screenshot, timing), Page Object integration, mixed parallel strategies, Allure/ExtentReports, CI/CD (GitHub Actions, Jenkins), or the 12-problem debugging guide. |
Playbook section index
| § | Section | Focus |
|---|---|---|
| 1 | Project Setup & Configuration | Maven + Surefire config |
| 2 | Suite XML Configuration | Multi-env, parallel, groups |
| 3 | BaseTest & Thread-Safe Driver | ThreadLocal, ConfigReader |
| 4 | Data Providers (Advanced) | Excel, JSON, CSV, parallel, cross-class |
| 5 | Factory Pattern | Cross-browser matrix |
| 6 | Listeners (Production Suite) | Retry, screenshot, timing |
| 7 | Soft Assertions & Dependencies | Groups, method deps |
| 8 | Page Object Integration | PageFactory, fluent POs |
| 9 | Parallel Execution Strategies | Method/class/test/mixed |
| 10 | Reporting Integration | Allure, ExtentReports |
| 11 | CI/CD Integration | GitHub Actions, Jenkins |
| 12 | Debugging Quick-Reference | 12 common problems |
| 13 | Best Practices Checklist | 14 items |
Limitations
- Use this skill only when the task clearly matches TestNG testing in Java and the local project context supports it.
- Verify commands, generated code, dependencies, credentials, and external service behavior before applying changes.
- Do not treat examples as a substitute for environment-specific tests, security review, or user approval for destructive or costly actions.
- This skill does not cover TestNG for Kotlin or non-JVM languages.
Related Skills
- junit-skill — JUnit 5 testing patterns (use when the project uses JUnit, not TestNG).
- maven-skill — Maven build lifecycle and plugin configuration (use when configuring Surefire/Failsafe beyond TestNG defaults).
- selenium-skill — Selenium WebDriver patterns (use alongside this skill for browser automation tests with TestNG).