Scala Learning

How to Generate Tests from a List of Inputs

I had a list of test cases that I had generated for my function. But I was too lazy to name each of them separately, especially since many of them were boundary tests (like 0 and -1) and not really branch tests that would each exercise one branch in your code. Adding a new named test for each one would duplicate a lot of code.

I tried avoiding that with one test that looped over the list of test cases. You can see that here (using a simple example of cube):

class MultipleAssertionsTry extends org.scalatest.FunSuite {
  // Defined here for testing.
  def cube(x: Int) = x * x * x

  val cubeTests = Seq(
    (0, 0),
    (1, 1),
    (-1, -1),
    (10, 1000),
    (9, 729),
  )

  test(s"cube test") {
    cubeTests foreach { case (input, output) =>
      assert(cube(input) == output)
    }
  }
}

When all the test cases passed, it worked correctly:

[info] MultipleAssertionsTry:
[info] - cube test (original)
...
[info] All tests passed.

However, when one test failed, it pointed it out but without mentioning the input argument:

[info] MultipleAssertionsTry:
[info] - cube test (original) *** FAILED ***
[info]   1000 did not equal 729 (MultipleAssertionsTry.scala:14)

I’d have to search through the list for a test case that expected 729 as the output.

And when multiple tests failed, it stopped after the first one. If I’d named the tests separately, it would have run each of them and showed me all the ways in which I had screwed up the function.

One solution to the above two problems is to generate a new named test for each test case (h/t this blog post):

  cubeTests foreach { case (input, output) =>
      test(s"cube of $input") {
        assert(cube(input) == output)
      }
  }

Now, when multiple tests fail, I get this:

[info] MultipleAssertionsTry:
[info] - cube test (original) *** FAILED ***
[info]   -1 did not equal 1 (MultipleAssertionsTry.scala:14)


[info] - cube of 0
[info] - cube of 1
[info] - cube of -1 *** FAILED ***
[info]   -1 did not equal 1 (MultipleAssertionsTry.scala:20)
[info] - cube of 10
[info] - cube of 9 *** FAILED ***
[info]   729 did not equal 728 (MultipleAssertionsTry.scala:20)

Nice.

Note that table-driven checks in ScalaTest didn’t do the trick because, even though they showed the input and output of the failed test, they stopped running tests after one failed, which was the key point.