thebushidocollective TestNG Parallel Execution
Master TestNG parallel test execution including thread pool configuration, suite-level parallelism, method-level parallelism, and thread safety patterns. This skill covers techniques for maximizing test throughput while maintaining test reliability.
Overview
TestNG supports parallel execution at multiple levels: suite, test, class, and method. Proper parallel configuration can significantly reduce test execution time, but requires careful consideration of thread safety and resource management.
Parallel Execution Modes
Suite-Level Parallelism
Run multiple <test> tags in parallel:
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Parallel Suite" parallel="tests" thread-count="3">
<test name="Chrome Tests">
<classes>
<class name="com.example.tests.BrowserTest"/>
</classes>
</test>
<test name="Firefox Tests">
<classes>
<class name="com.example.tests.BrowserTest"/>
</classes>
</test>
<test name="Safari Tests">
<classes>
<class name="com.example.tests.BrowserTest"/>
</classes>
</test>
</suite>
Class-Level Parallelism
Run test classes in parallel:
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Parallel Classes" parallel="classes" thread-count="4">
<test name="All Tests">
<classes>
<class name="com.example.tests.UserServiceTest"/>
<class name="com.example.tests.ProductServiceTest"/>
<class name="com.example.tests.OrderServiceTest"/>
<class name="com.example.tests.PaymentServiceTest"/>
</classes>
</test>
</suite>
Method-Level Parallelism
Run test methods in parallel:
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Parallel Methods" parallel="methods" thread-count="5">
<test name="Service Tests">
<classes>
<class name="com.example.tests.IndependentMethodsTest"/>
</classes>
</test>
</suite>
Instance-Level Parallelism
Run test instances in parallel (useful with Factory):
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Parallel Instances" parallel="instances" thread-count="3">
<test name="Factory Tests">
<classes>
<class name="com.example.tests.FactoryGeneratedTest"/>
</classes>
</test>
</suite>
Thread Pool Configuration
Basic Thread Configuration
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Thread Pool Suite" parallel="methods" thread-count="10">
<!-- Global thread pool configuration -->
<test name="Test Group 1" thread-count="5">
<!-- Override for this specific test -->
<classes>
<class name="com.example.tests.Test1"/>
</classes>
</test>
<test name="Test Group 2">
<!-- Uses suite-level thread-count -->
<classes>
<class name="com.example.tests.Test2"/>
</classes>
</test>
</suite>
Data Provider Parallel Execution
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="DataProvider Suite" data-provider-thread-count="20">
<test name="Data Driven Tests">
<classes>
<class name="com.example.tests.ParallelDataProviderTest"/>
</classes>
</test>
</suite>
public class ParallelDataProviderTest {
@DataProvider(name = "largeDataSet", parallel = true)
public Object[][] provideLargeDataSet() {
Object[][] data = new Object[100][2];
for (int i = 0; i < 100; i++) {
data[i] = new Object[]{"User" + i, "user" + i + "@example.com"};
}
return data;
}
@Test(dataProvider = "largeDataSet")
public void testWithParallelData(String name, String email) {
System.out.println(Thread.currentThread().getName() + " - Testing: " + name);
// Each data row runs in parallel
}
}
Thread Safety Patterns
Thread-Local Storage
import org.testng.annotations.*;
public class ThreadLocalTest {
// Thread-local storage for test-specific resources
private static ThreadLocal<WebDriver> driverThread = new ThreadLocal<>();
private static ThreadLocal<String> sessionThread = new ThreadLocal<>();
@BeforeMethod
public void setUp() {
// Initialize thread-local resources
driverThread.set(createWebDriver());
sessionThread.set(generateSessionId());
}
@AfterMethod
public void tearDown() {
// Clean up thread-local resources
WebDriver driver = driverThread.get();
if (driver != null) {
driver.quit();
}
driverThread.remove();
sessionThread.remove();
}
@Test
public void testParallelBrowser() {
WebDriver driver = driverThread.get();
String session = sessionThread.get();
System.out.println("Thread: " + Thread.currentThread().getName() +
" Session: " + session);
// Use thread-local driver
}
private WebDriver createWebDriver() {
// Create browser instance
return new ChromeDriver();
}
private String generateSessionId() {
return UUID.randomUUID().toString();
}
}
Synchronized Shared Resources
import org.testng.annotations.*;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.ConcurrentHashMap;
public class SynchronizedResourceTest {
// Thread-safe counter
private static AtomicInteger testCounter = new AtomicInteger(0);
// Thread-safe collection
private static ConcurrentHashMap<String, String> sharedCache = new ConcurrentHashMap<>();
// Lock object for critical sections
private static final Object lock = new Object();
@Test(threadPoolSize = 5, invocationCount = 100)
public void testAtomicOperations() {
int count = testCounter.incrementAndGet();
System.out.println("Test count: " + count);
}
@Test(threadPoolSize = 3, invocationCount = 50)
public void testConcurrentMap() {
String threadName = Thread.currentThread().getName();
sharedCache.put(threadName, String.valueOf(System.currentTimeMillis()));
// Thread-safe without explicit synchronization
}
@Test(threadPoolSize = 2, invocationCount = 10)
public void testSynchronizedBlock() {
synchronized (lock) {
// Critical section - only one thread at a time
performCriticalOperation();
}
}
private void performCriticalOperation() {
// Operations that require exclusive access
}
}
Immutable Test Data
import java.util.Collections;
import java.util.List;
import java.util.Arrays;
public class ImmutableDataTest {
// Immutable test data - inherently thread-safe
private static final List<String> TEST_USERS = Collections.unmodifiableList(
Arrays.asList("user1", "user2", "user3", "user4", "user5")
);
private static final Map<String, String> CONFIG = Collections.unmodifiableMap(
Map.of(
"url", "https://api.example.com",
"timeout", "30000",
"retries", "3"
)
);
@Test(threadPoolSize = 5, invocationCount = 20)
public void testWithImmutableData() {
// Safe to read from multiple threads
int userIndex = ThreadLocalRandom.current().nextInt(TEST_USERS.size());
String user = TEST_USERS.get(userIndex);
String url = CONFIG.get("url");
System.out.println(Thread.currentThread().getName() +
" - User: " + user + ", URL: " + url);
}
}
Test Isolation Patterns
Independent Test Methods
public class IndependentTestsExample {
// Each test method is completely independent
@Test
public void testFeatureA() {
// Create its own resources
UserService service = new UserService();
User user = service.createUser("testA");
assertNotNull(user);
// Clean up
service.deleteUser(user.getId());
}
@Test
public void testFeatureB() {
// Completely separate from testFeatureA
ProductService service = new ProductService();
Product product = service.createProduct("testB");
assertNotNull(product);
service.deleteProduct(product.getId());
}
@Test
public void testFeatureC() {
// No shared state with other tests
OrderService service = new OrderService();
Order order = service.createOrder();
assertNotNull(order);
service.cancelOrder(order.getId());
}
}
Isolated Database Tests
import org.testng.annotations.*;
public class IsolatedDatabaseTest {
private Connection connection;
private String testSchema;
@BeforeMethod
public void setUp() throws SQLException {
// Create isolated schema for each test
testSchema = "test_" + Thread.currentThread().getId() + "_" + System.currentTimeMillis();
connection = DriverManager.getConnection(DB_URL, USER, PASSWORD);
connection.createStatement().execute("CREATE SCHEMA " + testSchema);
connection.setCatalog(testSchema);
initializeTestData();
}
@AfterMethod
public void tearDown() throws SQLException {
// Drop isolated schema
connection.createStatement().execute("DROP SCHEMA " + testSchema + " CASCADE");
connection.close();
}
@Test
public void testDatabaseOperation1() throws SQLException {
// Operations in isolated schema
PreparedStatement ps = connection.prepareStatement(
"INSERT INTO users (name) VALUES (?)"
);
ps.setString(1, "TestUser1");
ps.executeUpdate();
ResultSet rs = connection.createStatement().executeQuery(
"SELECT COUNT(*) FROM users"
);
rs.next();
assertEquals(rs.getInt(1), 1);
}
@Test
public void testDatabaseOperation2() throws SQLException {
// Completely isolated from testDatabaseOperation1
PreparedStatement ps = connection.prepareStatement(
"INSERT INTO products (name) VALUES (?)"
);
ps.setString(1, "TestProduct");
ps.executeUpdate();
}
private void initializeTestData() throws SQLException {
// Create tables in isolated schema
}
}
Parallel Execution with Dependencies
Preserving Order Within Groups
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Ordered Parallel" parallel="classes" thread-count="3">
<test name="Ordered Test" preserve-order="true">
<classes>
<!-- Classes run in parallel, methods in order -->
<class name="com.example.tests.OrderedTest1">
<methods>
<include name="step1"/>
<include name="step2"/>
<include name="step3"/>
</methods>
</class>
<class name="com.example.tests.OrderedTest2"/>
</classes>
</test>
</suite>
Group Threading
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Group Threading" parallel="methods" thread-count="4" group-by-instances="true">
<test name="Instance Grouped">
<classes>
<class name="com.example.tests.InstanceGroupTest"/>
</classes>
</test>
</suite>
public class InstanceGroupTest {
private String instanceId;
@Factory
public Object[] createInstances() {
return new Object[] {
new InstanceGroupTest("instance1"),
new InstanceGroupTest("instance2"),
new InstanceGroupTest("instance3")
};
}
public InstanceGroupTest() {}
public InstanceGroupTest(String instanceId) {
this.instanceId = instanceId;
}
@Test
public void step1() {
System.out.println(instanceId + " - Step 1");
}
@Test(dependsOnMethods = "step1")
public void step2() {
System.out.println(instanceId + " - Step 2");
}
@Test(dependsOnMethods = "step2")
public void step3() {
System.out.println(instanceId + " - Step 3");
}
}
Performance Optimization
Optimal Thread Count
public class ThreadCountOptimization {
// Determine optimal thread count based on available resources
public static int getOptimalThreadCount() {
int availableProcessors = Runtime.getRuntime().availableProcessors();
// For CPU-bound tests
int cpuBoundThreads = availableProcessors;
// For I/O-bound tests (network, file, database)
int ioBoundThreads = availableProcessors * 2;
// For mixed workloads
int mixedThreads = (int) (availableProcessors * 1.5);
return mixedThreads;
}
}
Resource Pool Pattern
import java.util.concurrent.*;
public class ResourcePoolTest {
// Connection pool for parallel tests
private static BlockingQueue<Connection> connectionPool;
@BeforeSuite
public void setUpSuite() {
int poolSize = 10;
connectionPool = new ArrayBlockingQueue<>(poolSize);
for (int i = 0; i < poolSize; i++) {
connectionPool.offer(createConnection());
}
}
@AfterSuite
public void tearDownSuite() {
Connection conn;
while ((conn = connectionPool.poll()) != null) {
closeConnection(conn);
}
}
@Test(threadPoolSize = 5, invocationCount = 50)
public void testWithPooledConnection() throws InterruptedException {
Connection conn = connectionPool.take(); // Borrow
try {
// Use connection
performDatabaseOperation(conn);
} finally {
connectionPool.offer(conn); // Return
}
}
private Connection createConnection() {
// Create database connection
return null;
}
private void closeConnection(Connection conn) {
// Close connection
}
private void performDatabaseOperation(Connection conn) {
// Database operations
}
}
Reporting for Parallel Tests
Custom Reporter for Parallel Execution
import org.testng.*;
import java.util.concurrent.ConcurrentHashMap;
public class ParallelTestReporter implements ITestListener {
private static ConcurrentHashMap<Long, List<String>> threadTestMap =
new ConcurrentHashMap<>();
@Override
public void onTestStart(ITestResult result) {
long threadId = Thread.currentThread().getId();
threadTestMap.computeIfAbsent(threadId, k -> new CopyOnWriteArrayList<>())
.add(result.getName());
}
@Override
public void onFinish(ITestContext context) {
System.out.println("\n=== Thread Distribution Report ===");
threadTestMap.forEach((threadId, tests) -> {
System.out.println("Thread " + threadId + ": " + tests.size() + " tests");
tests.forEach(test -> System.out.println(" - " + test));
});
System.out.println("Total threads used: " + threadTestMap.size());
}
}
Timeout Configuration
public class TimeoutTest {
@Test(timeOut = 5000)
public void testWithTimeout() {
// Fails if takes more than 5 seconds
}
@Test(timeOut = 10000, threadPoolSize = 3, invocationCount = 10)
public void testParallelWithTimeout() {
// Each invocation has 10 second timeout
}
}
<!DOCTYPE suite SYSTEM "https://testng.org/testng-1.0.dtd">
<suite name="Timeout Suite" time-out="60000">
<!-- Suite-level timeout: 60 seconds total -->
<test name="Quick Tests" time-out="10000">
<!-- Test-level timeout: 10 seconds for all tests in this group -->
<classes>
<class name="com.example.tests.QuickTest"/>
</classes>
</test>
</suite>
Best Practices
- Design for independence - Tests should not depend on shared mutable state
- Use ThreadLocal for per-thread resources - Drivers, sessions, connections
- Prefer immutable data - Thread-safe by design
- Set appropriate thread counts - Based on resource availability
- Implement proper cleanup - Prevent resource leaks in parallel execution
- Use thread-safe collections - ConcurrentHashMap, CopyOnWriteArrayList
- Configure timeouts - Prevent hung tests from blocking threads
- Monitor thread distribution - Ensure balanced workload
- Test locally first - Verify thread safety before CI/CD
- Document thread safety requirements - Clear expectations for test authors
Common Pitfalls
- Shared mutable state - Causes race conditions and flaky tests
- Static fields without synchronization - Not thread-safe
- Resource contention - Too many threads competing for limited resources
- Order dependencies - Tests that assume execution order
- Missing cleanup - ThreadLocal resources not removed
- Insufficient isolation - Database tests affecting each other
- Too many threads - Overhead exceeds benefits
- Ignoring timeouts - Hung tests blocking execution
- Non-deterministic failures - Hard to reproduce parallel issues
- Improper connection pooling - Connection leaks or exhaustion
When to Use This Skill
- Reducing test suite execution time
- Configuring CI/CD parallel test execution
- Implementing thread-safe test infrastructure
- Designing parallel-friendly test architecture
- Troubleshooting parallel test failures
- Optimizing resource utilization in tests
- Building scalable test frameworks
- Implementing cross-browser parallel testing