Quata 3: Asserting in Unit Tests

feature image

Quata 3: Asserting in Unit Tests

Unit tests are, in short, safety nets for your code. There are only two possible reasons why a test breaks:

  • It needs to be updated to reflect the new code (syntax or behavior).
  • It is now broken because an unintentional functional change was introduced in the code.

On the other hand, when a functional change is intentional, but none of the unit tests break, either of the following is true:

  • The code is not covered.
  • The unit test(s) are not correctly written.

In this quata, I’ll present one half of my definition of “correct” when it comes to unit tests. I’ll cover the other half in Quata 4.

Consider the following assertions against a method that splits a string into a list of uppercase words.

List<String> words = myObj.splitAndCap("code quality mindset");
assertEquals(words.size(), 3);

try {
  myObj.splitAndCap("code quality mindset");
} catch (MyException e) {
  fail("exception not expected!");

try {
} catch (MyException e) {
  assertTrue(e.getMessage().contains("illegal argument"));
  assertEquals(IllegalArgumentException.class, e.getCause().getClass());

Here are my comments:

Line #Comment
2The parameters are switched. The assertEquals() method accepts the expected value and the actual value, in that order. Same thing with assertSame(). The order especially matters when the test fails. It will tell you which value it is expecting and which value it actually got. When they’re switched like this, the error message would confuse you.
3It’s usually not enough to assert that something is not null. If you know the expected value, then do assertEquals() or assertSame(). That way, when a bug is introduced that changes the value, your test would fail. assertNotNull() would keep passing. And that’s not right.
4If this assertion fails, it won’t tell you what the actual value was. When assertEquals() fails, it will. So, prefer it over assertTrue() wherever applicable.
5Wait, line 5 is a blank line. Well, it shouldn’t have been. We’re missing one more assertion. The test forgot about the third word! The code can mess up the third word and the test would still pass.

By the way, if your answer was, there’s a way to compare two lists in one shot, you get bonus points. I wanted to show different examples of incorrect asserts. Hence, the individual asserts on each element.
9This is redundant. There is no need to fail() here. In fact, we don’t need the entire try-catch business. It’s not technically incorrect, but it’s just not needed. Just let it throw the exception. JUnit will treat it as a failure, which is what you want.
14It’s the opposite situation here. We need a fail() on this line. We are expecting an exception and want to verify that the cause is an IllegalArgumentException. If someone introduces a bug that stops it from throwing that exception, it would not get to the catch block. Your test then automatically passes, when it should not. The fail() on this line would basically assert that “no, it shouldn’t get this far. It should go to the catch block instead.”
15assertTrue() comes in handy when you do not know the entire value, in which case, you tend to use methods such as String.contains() or String.startsWith(). However, when it fails, you wouldn’t know why. It would be prudent to pass the entire value as the first parameter to assertTrue(). It would show up in the error message when it fails, helping you figure out why the contains() or the startsWith() returned false.

Having said that, you should ask yourself whether you really need this assertion. I usually do not assert against exception’s messages, given how they’re pretty arbitrary. Over-asserting leads to fragile tests. If someone changes the exception message, I probably don’t want it to break my test. But of course, this is on a case-by-case basis. There may be cases where the exception message does matter and therefore should be asserted against.

Here’s the corrected code:

List<String> words = myObj.splitAndCap("code quality mindset");
assertEquals(3, words.size());
assertEquals("CODE", words.get(0));
assertEquals("QUALITY", words.get(1));
assertEquals("MINDSET", words.get(2));

// removed the unnecessary try-catch block around this call
myObj.splitAndCap("code quality mindset");

try {
  fail("exception expected!");
} catch (MyException e) {
  // kept the line below assuming the message really matters;
  // note the first parameter;
  // ask yourself if you really need this assertion though.
  assertTrue(e.getMessage(), e.getMessage().contains("illegal argument"));
  assertEquals(IllegalArgumentException.class, e.getCause().getClass());

It’s time to present one half of my definition of “correct”:

A unit test is correct if it asserts everything that matters, no more, no less.

Under-asserting makes for incomplete tests. Think huge holes in the safety net, if you’d still call it that. I wouldn’t feel safe having that under me, would you? Changing something in the code that matters should fail the test. The above example failed to verify the value of the third entry in the array. That was a huge hole in the net.

Over-asserting, on the other hand, makes for fragile tests. In the above example, verifying the exception message could be considered over-asserting. If the exception message doesn’t really matter, your test should not verify it. Otherwise, any simple change in the message could unnecessarily break your test.

You will find the other half of my definition in the next quata.

Leave a Reply

Your email address will not be published. Required fields are marked *

Post navigation

Previous Post :