Multiple Threads Testing Idioms In Java

Today I needed to make a few modification to one of our older Java applications. So, first thing’s first – I checked out the latest copy and ran the test suite. Surprisingly enough all the tests passed. But, something wasn’t right. I caught the glimpse of a stack trace flipping by in Eclipse’s console window. Scrolling up to it I found out an AssertionError was thrown. How come the tests passed?

A little digging revealed this (simplified) test case:

    public void buggieTest() throws Exception {
        // Do some setup
        // ...

        new Thread() {
            @Override
            public void run() {
                int x = 0;
                // Check a few things that change x
                // ...
                assertTrue("An error", x > 3); // This fails!
            }
        }.start();

        // Wait for other thread
        Thread.sleep(2000);
    }

Can you spot the bug? The assertion fails, but in another thread. The test is testing an asynchronous module’s response. JUnit’s test executer isn’t aware of threads other than its own, so it thinks everything is OK. For the sake of the original developers I’ll mention the fact that they were using a home-built test runner that set ThreadGroups in order to catch those exceptions, but nowadays everyone use Eclipse’s JUnit plugin in my work place and that logic is no more.

So, what can we do? I first considered passing a boolean between the two threads so that the test’s thread would fail() if the other thread indicated failure. The problem is that the original error is either lost or hard to show.

After some thinking (and because I’m currently reading the great Concurrent Programming in Java book) I came up with this simple solution using the JDK Exchanger class:

public void betterTest() throws Exception {
    // Do some setup
    // ...

    final Exchanger<Exception> exchanger = new Exchanger<Exception>();

    new Thread() {
        @Override
        public void run() {
            Exception thrown = null;
            try {
                int x = 0;
                // Check a few things that change x
                // ...
                assertTrue("An error", x > 3); // This fails!
            } catch (Exception e) {
                thrown = e;
            }
            try {
                exchanger.exchange(thrown);
            } catch (InterruptedException ignored) {}
        }
    }.start();

    // Wait for other thread
    Exception exception = exchanger.exchange(null);
    if (exception != null) {
        throw exception;
    }
}

So, what’s so better about this more complicated version? Well, first of all, it works. We use the exchanger to pass the exception to the testing thread. Another nice benefit is the fact we’re throwing the original exception – meaning that Eclipse’s plugin shows the right stack trace and clicking it gets you to the actual line that failed and not a simple fail() as opposed to the option I mentioned before.

Do you know of a better way to do this? I’m trying to think whether something can be added to JUnit for this purpose. Anyway, this results in a cool test that works great with the different plugins.

Happy testing!

Advertisements

One response to “Multiple Threads Testing Idioms In Java

  1. Nice. We had similar issues with some of the exercises of our Java Specialist Master Course. There is a bug (in my opinion) in the Condition.await() method, in that it does not mirror the behavior of the Object.wait() method accurately. If you stop the thread whilst it is doing await(), the thread does not pick up the lock again. Thus, when you unlock in finally, you end up with IllegalMonitorStateExceptions.

    The point is, we saw the same thing go wrong. It appears to me that JUnit does not wait for the other threads to complete even, because we don’t always see the output from all of these threads.

    Anyway, it might also be the IntelliJ plugin that causes this, I have not had the time to investigate this further.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s