How should I unit test multithreaded code?

What is Multithreaded Code?

Multithreaded code is code that uses multiple computing threads to execute a program. It is a model of software code execution that allows for multiple threads to be created within a process that runs independently but concurrently shares resources. In this case, a thread is the smallest task that can be processed independently from a software code. Multithreaded code can either be running concurrently,  in parallel, or both.

Concurrency occurs when multiple threads are making progress for overlapping periods but not necessarily simultaneously, for example, in a single-core CPU that switches quickly between tasks. In layman’s terms, concurrency is analogous to a single librarian checking in returned books while answering a visitor’s inquiry about book locations. Without concurrency, the librarian would have to finish checking in all the books before being able to assist the visitor with their inquiry.

On the other hand, parallelism means the tasks are running simultaneously, and it requires a computing system with multiple CPU cores and physical threads. Using our librarian example, parallelism is analogous to having multiple librarians working simultaneously, each performing a different task.

As Rob Pike said, “Concurrency is about dealing with many things at once. Parallelism is about doing lots of things at once.”

Why is it difficult to unit test Multithreaded Code?

While fast to execute, multithreaded code tends to have qualities that make it harder to debug and unit test than sequential code. The main challenge is non-determinism, which results in code issues like race conditions and deadlocks. Non-determinism implies that given the same code and shared resources, the program can follow different execution paths, resulting in different outputs or no results.

Race conditions occur when multiple threads access a shared resource and try to change it simultaneously, resulting in unpredictable results. It manifests in a system when two or more operations must execute in a specific sequence, but the system implements them in the wrong order. For example, imagine two threads changing a value x, equal to 100, by subtracting 10 simultaneously. If they run precisely simultaneously, they will read x as equal to 100, resulting in an answer of 90. However, the result is expected to be 80 since x should be subtracted twice.

On the other hand, a deadlock happens when threads wait indefinitely because each thread is waiting for the other to release a lock on resources.

public class BankAccountRace {
    private int balance = 100;

    public void decrementBalance() {
        int newBalance = balance - 10;
        try {
            Thread.sleep(100); // add a small delay to create a race condition
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        balance = newBalance;
    }

    public int getBalance() {
        return balance;
    }

    public static void main(String[] args) throws InterruptedException {
        BankAccountRace account = new BankAccountRace();

        Thread threadA = new Thread(() -> {
            account.decrementBalance();
        });

        Thread threadB = new Thread(() -> {
            account.decrementBalance();
        });

        threadA.start();
        threadB.start();

        threadA.join();
        threadB.join();

        System.out.println("Final Balance: " + account.getBalance());
    }
}

Best Practices for Unit Testing Multithreaded Code in Java

To test multi-threaded code and applications, best practices for code development and unit testing should be implemented. Testing multithreaded code can be implemented using the following methods:

Adopt Java Concurrency Utilities

Java provides developers with multiple high-level concurrency utilities useful in coordinating thread execution during unit testing. The principal utilities include Semaphore, CountDownLatch, and CyclicBarrier. Semaphore is a synchronization aid that allows a fixed number of threads to access a shared resource simultaneously. It maintains a set of permits acquired and released by threads and blocks until a permit is available. CountDownLatch is a synchronization aid that allows one or more threads to wait until a set of operations being performed in other threads completes. It maintains a decremented count each time an operation completes and blocks until the count reaches zero. CyclicBarrier is a synchronization utility that allows a set of threads to wait for each other to reach a common barrier point. It maintains a decremented barrier count each time a thread arrives at the barrier and blocks until the count reaches zero.

import java.util.concurrent.CountDownLatch;
import java.util.concurrent.CyclicBarrier;
import java.util.concurrent.Semaphore;

public class BankAccountUtils {
    public static void decrementBalance(BankAccount account, CountDownLatch latch) {
        int newBalance = account.getBalance() - 10;
        try {
            Thread.sleep(100);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        account.setBalance(newBalance);
        latch.countDown();
    }

    public static void incrementBalance(BankAccount account, CountDownLatch latch) {
        int newBalance = account.getBalance() + 10;
        try {
            Thread.sleep(100);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
        account.setBalance(newBalance);
        latch.countDown();
    }

    public static void transferBalance(BankAccount fromAccount, BankAccount toAccount, Semaphore semaphore) {
        try {
            semaphore.acquire();
            int amount = fromAccount.getBalance();
            fromAccount.setBalance(fromAccount.getBalance() - amount);
            toAccount.setBalance(toAccount.getBalance() + amount);
        } catch (InterruptedException e) {
            e.printStackTrace();
        } finally {
            semaphore.release();
        }
    }

    public static void main(String[] args) throws Exception {
        BankAccount accountA = new BankAccount(100);
        BankAccount accountB = new BankAccount(100);

        CountDownLatch latch = new CountDownLatch(2);
        CyclicBarrier barrier = new CyclicBarrier(2);
        Semaphore semaphore = new Semaphore(1);

        Thread threadA = new Thread(() -> {
            decrementBalance(accountA, latch);
            try {
                barrier.await();
            } catch (Exception e) {
                e.printStackTrace();
            }
        });

        Thread threadB = new Thread(() -> {
            incrementBalance(accountB, latch);
            try {
                barrier.await();
            } catch (Exception e) {
                e.printStackTrace();
            }
        });

        Thread threadC = new Thread(() -> {
            transferBalance(accountA, accountB, semaphore);
        });

        threadA.start();
        threadB.start();
        threadC.start();

        latch.await();
        barrier.await();
        semaphore.acquire();

        System.out.println("Final Balance A: " + accountA.getBalance());
        System.out.println("Final Balance B: " + accountB.getBalance());

        semaphore.release();
    }
}

Isolating concurrency logic

When designing the code, try to make the code that handles concurrency, like thread creation and synchronization, separate from the code that creates the results (business logic). This ensures the business logic and the threads code can be unit tested in isolation without involving the threads.

Mocking

This involves simulating thread behaviors by using mocking frameworks like Mockito or EasyMock. For example, suppose we have a BankAccountMock class that uses a TransactionLog object to log transactions. We can use mocking to simulate the behavior of the TransactionLog object in our unit tests without writing to an actual log file.

import static org.mockito.Mockito.*;

import java.util.concurrent.CompletableFuture;

import org.junit.Test;

public class BankAccountMock {
    @Test
    public void testDecrementBalance() throws Exception {
        // create a mock transaction log
        TransactionLog mockLog = mock(TransactionLog.class);

        // create a bank account with the mock transaction log
        BankAccounts account = new BankAccounts(mockLog);

        // create a completable future that returns a balance of 90
        CompletableFuture<Integer> future = CompletableFuture.completedFuture(90);

        // mock the transaction log to return the completable future
        when(mockLog.logTransaction(anyString())).thenReturn(future);

        // decrement the balance
        CompletableFuture<Integer> result = account.decrementBalance();

        // verify that the transaction log was called with the correct parameters
        verify(mockLog).logTransaction("debit:10");

        // verify that the result is 90
        assert(result.get() == 90);
    }
}

Logging

Incorporate detailed logging into your code. This can help you trace and understand the sequence of events in a multithreaded environment, especially when debugging failing tests.

Use Testing Libraries

Leverage unit testing frameworks and libraries like Junit for writing tests. Furthermore, use specialized libraries like JMH to test performance and load.

import org.junit.Test;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import static org.junit.Assert.assertEquals;


public class BankAccountTest {
    @Test
    public void testConcurrentDecrementBalance() throws InterruptedException {
        int numberOfThreads = 10;
        CountDownLatch latch = new CountDownLatch(numberOfThreads);
        BankAccounts account = new BankAccounts();
        ExecutorService service = Executors.newFixedThreadPool(numberOfThreads);

        for (int i = 0; i < numberOfThreads; i++) {
            service.submit(() -> {
                account.decrementBalance();
                latch.countDown();
            });
        }

        // Wait for all threads to finish, but no longer than 2 seconds.
        latch.await(2, TimeUnit.SECONDS);
        service.shutdown();

        // If there are 10 threads and each one decrements the balance by 10, the final balance should be 0.
        assertEquals(0, account.getBalance());
    }
}

class BankAccounts {
    private int balance = 100;

    public synchronized void decrementBalance() {
        balance = balance - 10;
    }

    public int getBalance() {
        return balance;
    }
}

The list of best practices is not exhaustive. Other practices like stress testing the code, implementing timeouts, and checking for thread interruptions should be considered when unit testing code. In conclusion, Java offers utilities for writing and testing multithreaded code, which should be utilized during the software development lifecycle.