Software QA FYI - SQAFYI

Testing with Scala


When learning a new language or framework one of my early questions is nowadays: What about testing? Obviously I asked that for Scala as well.

If there is anything to complain about I guess it is once more the large number of options one has.

Scala is highly compatible with Java and this includes the popular testing framework Junit. You can just write your tests using JUnit as you are used to but replacing Java classes with Scala classes
view source
01 class ConnectionTest {
02 import org.junit.Test
03 import org.junit.Assert._
05 @Test
06 def connection_Equals_itself() {
07 val con = Connection(Corner(), Corner())
08 assertEquals(con, con)
09 assertEquals(con.hashCode, con.hashCode)
10 }
11 }

This works nice. But I promise once you get used to some Scala goodness youíll want to use it inside your tests as well. So time to look at some Scala test framework. There are two major test frameworks for Scala: Specs and ScalaTest.

I picked ScalaTest because it allows for lots of different testing styles. So lots of opportunities for experimentation. The style I prefer so far is based on Ďfeaturesí. Very differently from JUnit single tests arenít defined in separate methods, but all tests are placed directly inside the body of the class and thus are executed once the class is instantiated. (Judging by the API there are ways to fine tune that, but I havenít explored them yet.)

Tests are grouped in features and each test is named scenario. Since tests arenít methods but method calls you canít use the method name as the name of the test. Instead you provide a String parameter. This allows for loooong descriptive names. At least I consider this a good thing. One feature spec of a little project I am working on looks like this.

view source
01 class ImageComparisonTest extends FeatureSpec {
03 feature("a user can compare the screenshot of a swing component with
an image stored on disk") {
04 scenario("comparing a component to a NOT matching image") {
05 pending
06 }
07 scenario("comparing a component to a NOT existing image in headl
ess mode") {
08 pending
09 }
10 scenario("comparing a component to a NOT existing image in withhead mode") {
11 pending
12 }
13 }
14 }

The Trait FeatureSpec makes the feature and scenario methods available. The Ďpendingí statements mark the test as .. well .. pending because neither the test nor the code is implemented yet. Pending tests are reported as ĎIgnoredí. The simplest way to make assertions in a test is to call assert or other methods of the Assertions object. But one can also mix in various traits for different styles of specifying assertions. For example ShouldMatchers provide a nice DSL allowing statements like this:

view source
1 (5 * 23) should equal (115)

This stile of specifying assertions also guarantees really nice error messages in the case of a failing assertion. This test

view source
01 class SomeTest extends FeatureSpec with ShouldMatchers
02 feature ("ShouldMatcher provide nice error messages"){
03 scenario ("Making a correct assertion"){
04 (5 * 23) should equal (115)
05 }
06 scenario ("Making a stupid assertion"){
07 (5 * 23) should equal (116)
08 }
09 }
10 }

results in this error message when executed:
view source
1 [error] Test Failed: Feature: ShouldMatcher provide nice error messages Making a stupid assertion
2 org.scalatest.TestFailedException: 115 did not equal 116
3 at org.scalatest.matchers.Matchers$class.newTestFailedException(Matchers.scala:148)

In JUnit tests I sometimes have empty lines or even comments in my tests in order to visually seperate the setup the actual action I want to test and the assertion. With ScalaTest you have a much nicer way to do that: the GivenWhenThen trait. With it you donít put comments in your code, but method calls with a string parameter, which once again will end up in the output of your test:

view source
01 class SomeTest extends FeatureSpec with ShouldMatchers with GivenWhenThen
02 feature ("ShouldMatcher provide nice error messages"){
03 scenario ("Making a correct assertion"){
04 given ("a number")
05 val number = 5
06 and ("an other number")
07 val otherNumber = 23
08 when ("multiplying one with the other")
09 val result = number * otherNumber
10 and ("the other way round")
11 val otherResult = otherNumber * number
12 then ("the results will be the same")
13 result should equal (otherResult)
14 }
15 }
16 }

While in this trivial case and with the weak syntax highlighting of my blog this might be a little distraction, it is really a nice form of in line commenting tests.

Of course you donít just write test, you want them executed as well. There are Runners for JUnit and TestNG so you can execute your ScalaTests with the testing framework you are used to. But I absolutely recommend the use of sbt to execute your tests directly. Sbt is a simple build tool. Setup is extremely easy, you start it enter the command ~test and it will run all your tests whenever you save your code. Really nice, especially since it is much faster then the scala compilers inside the IDEs I have seen so far.

Full article...

Other Resource

... to read more articles, visit

Testing with Scala