These notes will introduce you to software testing and how to accomplish it in Java using the JUnit framework. These are older notes using JUnit 4 in case you have problems with JUnit 5.
Obviously it is important to test software before releasing it, to iron out bugs. Software testing can be done in an informal, ad-hoc way, however the disadvantage of this is that the developer is likely to miss out testing crucial functionality. The robustness of the software can be enhanced by taking a more formal approach to testing, by drawing up a test plan documenting all tests and the expected and actual output, as well as performing a series of unit tests designed to test different parts of the system.
A unit test is designed to test one small part of the system in isolation, such as a method. Unit tests are written to test different outcomes of a method. For example, a Product class (used in shop management software) might have a sell() method which takes the number of items to sell as a parameter. This could have three outcomes:
public class Product
{
// ... rest of class omitted
static final int SUCCESS = 0, INSUFFICIENT_QUANTITY = 1, INVALID_REQUESTED_QUANTITY = 2;
public int sell (int amount)
{
if(quantity
Unit tests for this sell() method could involve:
- Making a valid sale and checking that SUCCESS is returned;
- Making a valid sale and checking that the amount is reduced by the expected amount;
- Supplying an invalid amount (0 or less), checking that INVALID_REQUESTED_QUANTITY is returned;
- Supplying an invalid amount (0 or less), checking that the quantity in stock does not change;
- Supplying an amount greater than the quantity in stock,
checking that INSUFFICIENT_QUANTITY is returned;
- Supplying an amount greater than the quantity in stock
and checking that the quantity in stock does not change.
JUnit - A Unit Testing Framework
In Java, unit testing is made straightforward by the open-source unit testing framework
JUnit. This is automatically available as part of Eclipse, so is quite easy to use.
There is a good tutorial at Vogella which was partly
used to research these notes.
Example
Imagine you have an Item class:
public class Item {
String name;
double price;
int qty;
public Item (String name, double price, int qty)
{
this.name=name;
this.price=price;
this.qty=qty;
}
public String getName()
{
return name;
}
public double getPrice()
{
return price;
}
public int getQuantity()
{
return qty;
}
public boolean isInStock()
{
return qty>0;
}
public boolean sell()
{
if(isInStock())
{
qty--;
return true;
}
return false;
}
public boolean restock(int qty)
{
if(qty>0)
{
this.qty += qty;
return true;
}
return false;
}
public String toString()
{
return name +" " +price +" " +qty;
}
}
To use JUnit, we have to create a test class with a series of unit tests.
Here is an example of a series of unit tests that you would run with JUnit on the Item class above:
import static org.junit.Assert.*;
import org.junit.Test;
public class ItemTestBasic {
public ItemTestBasic()
{
}
@Test
public void testSellOnceInStock()
{
Item i = new Item("Mars bar",0.60, 1);
assertEquals( true, i.sell() );
}
@Test
public void testSellOnceNotInStock()
{
Item i = new Item("Mars bar",0.60, 0);
assertEquals( false, i.sell() );
}
@Test
public void testIsInStockNeg() {
Item i = new Item("mars bar", 0.60, -1);
assertEquals( false, i.isInStock() );
}
@Test
public void testIsInStockZero() {
Item i = new Item("mars bar", 0.60, 0);
assertEquals( false, i.isInStock() );
}
@Test
public void testIsInStockPositive() {
Item i = new Item("mars bar", 0.60, 1);
assertEquals( true, i.isInStock() );
}
@Test
public void testSellTwiceNotInStock()
{
Item i = new Item("Mars bar",0.60, 1);
i.sell();
assertEquals( false, i.sell() );
}
@Test
public void testRestock() {
int amount=3, initial=10;
Item i = new Item("Mars bar", 0.60, initial);
i.restock(amount);
assertEquals( initial+amount,i.getQuantity());
}
}
Looking at this in more detail:
- Unless they are parameterised (see later), test classes must have a no-arguments constructor (ItemTestBasic() here)
- Note how we have a series of test methods. Note how each is preceded with the annotation
@Test. This informs JUnit that the method coming up is a test that it should run.
(This is actually Java syntax, not specifically JUnit; for more on Java
annotations see here).
- Note how each method creates an instance of the class under test (Item here),
performs an operation on it (e.g. sells it) and tests whether a condition has been met
with assertEquals(). assertEquals() is what is called an assertion: a general
technique in debugging to check whether a condition has been met. It takes two parameters:
- The expected result, e.g. true or false;
- The method under test.
- We'll now illustrate this by going through the testSellOnceInStock() test.
This tests that we can sell an item that is in stock once, successfully. We:
- create a new item that is in stock:
Item i = new Item("Mars bar",0.60, 1);
- test that sell() returns true:
assertEquals( true, i.sell() );
- If assertEquals() passes, i.e. i.sell() does return true, then the test will pass.
If on the other hand i.sell() returns false, then the assertion will fail because i.sell()
is not returning the expected value, and so the test will fail
- This first test tests that we can successfully perform an operation. However we can also write tests to prove
that an expected error condition does happen. The second test, testSellOnceNotInStock(), does
this:
@Test
public void testSellOnceNotInStock()
{
Item i = new Item("Mars bar",0.60, 0);
assertEquals( false, i.sell() );
}
Note how we are asserting that i.sell() returns false. The test will pass
if it does return false, and fail if it returns true: note again we are
testing the expected outcome. Here, since the initial quantity is 0, the expected outcome
is that the i.sell() method will return false because the item is out of stock.
- The next three tests test the isInStock() method in three different situations:
negative stock, zero stock and positive stock. We test that the first two conditions return false
and the last condition returns true.
- Try making a test fail. Change the isInStock() method in Item to:
public boolean isInStock()
{
return qty>=0; // greater than or equal to, not greater than
}
This method is now wrong because an item should be out of stock if the quantity is 0 or less. Quantities of
0 do not count as being in stock! Consequently the testIsInStockZero() test will now fail.
The assertEquals() asserts that isInStock() returns false. Now, however it will return true, so the
assertion and consequently the test will fail.
- The next test, testSellTwiceNotInStock(),
is slightly more complex, it tries to sell an item twice if the initial quantity in stock is one.
The expected outcome is that we will be able to sell it once, then it will be unable to sell
(sell() returns false) if we try to sell it again. We sell it once
without testing, then assert that the second call to sell() returns false.
- The last test, testRestock(), shows that we are not limited to testing methods with a boolean
return value. We can test any return value of a method.
It creates an item with a given initial quantity, calls restock() with an amount to
increase the stock by, and then checks, with an assertion, that the getQuantity() method returns the sum of the initial
value and the amount to increase the stock by. In doing so, we check that the restock() method has
restocked the item by the specified amount.
- Note how we are performing a wide range of tests and focusing in particular on boundary conditions: for instance
we check that zero, as well as negative numbers, are treated as out-of-stock conditions. Boundary conditions are
where many programming bugs occur.
Parameterised Tests
The first example performed three very similar tests: testing that the isInStock() method gives the expected
result for the values -1, 0 and 1. Clearly this is rather inefficient. It would be better if we could run a single test with
one or more parameters - and luckily we can. The example below has a single test, testIsInStockMultipleValues()
and runs it three times, passing the parameters -1, then 0, then 1 to it:
import static org.junit.Assert.*;
import org.junit.Test;
import java.util.Arrays;
import java.util.Collection;
import org.junit.runners.Parameterized.Parameters;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
@RunWith(Parameterized.class)
public class ParamItemTest {
int parameter;
public ParamItemTest(int parameter)
{
this.parameter=parameter;
}
@Parameters
public static Collection data()
{
Object[][] data = { { -1 } , { 0 } , {1 } } ;
return Arrays.asList(data);
}
@Test
public void testIsInStockMultipleValues()
{
Item i = new Item("Mars bar",0.60, parameter);
assertEquals( parameter>0, i.isInStock() );
}
}
Note the following:
- The class is preceded with the annotation
@RunWith(Parameterized.class). This tells JUnit to run the
class with the parameterised tester, rather than the normal tester. Consequently we also need various additional
import statements.
- Parameterised test classes must take at least one parameter in their constructor. This is the parameter which will
be passed to the tests.
- Note the data() method:
@Parameters
public static Collection data()
{
Object[][] testData = { { -1 } , { 0 } , {1 } } ;
return Arrays.asList(testData);
}
This is the key method to running parameterised tests. The data() method returns a list of
parameters to test: the test class will be instantiated with each parameter in the list in turn.
The data: Object[][] data = { { -1 } , { 0 } , {1 } } ; is a two-dimensional array
of parameters. Each member of the array testData corresponds to the parameters to test each time, but
each member is itself an array so that we can send multiple parameters to the test
(we'll see this in the next example). For now, however, we're only sending one parameter to the test so each
member of testData is an array with one member - the parameter to test.
- What JUnit will then do is instantiate the ParamItemTest class with each parameter from the
testData array in turn. Note how the actual test method, testIsInStockMultipleValues(),
uses the parameter we passed into the constructor to do the test. We create an Item with a stock level of
the parameter, and then test isInStock() for the expected result, which will be the result of the
condition
parameter>0
.
Parameterised Tests with Multiple Parameters
The following example is a modification of the previous one, which takes multiple parameters per test run:
import static org.junit.Assert.*;
import org.junit.Test;
import java.util.Arrays;
import java.util.Collection;
import org.junit.runners.Parameterized.Parameters;
import org.junit.runner.RunWith;
import org.junit.runners.Parameterized;
@RunWith(Parameterized.class)
public class ParamItemTest2 {
int paramAmount;
boolean paramExpectedResult;
public ParamItemTest2(int paramAmount, boolean paramExpectedResult)
{
this.paramAmount=paramAmount;
this.paramExpectedResult=paramExpectedResult;
}
@Parameters
public static Collection data()
{
Object[][] testData = { { -1, false } , { 0,false } , {1,true } } ;
return Arrays.asList(testData);
}
@Test
public void testIsInStockMultipleValues()
{
Item i = new Item("Mars bar",0.60, paramAmount);
assertEquals( paramExpectedResult , i.isInStock() );
}
}
Note how each element in the testData array is now a two-member array, representing the
two parameters that are passed into the test class: the initial amount in stock and the expected result of
isInStock() for that initial amount. Each pair of values in the testData array will be passed in
turn into the constructor, which now takes two parameters corresponding to these two values.
Note how the test method uses these parameters to perform the test: it instantiates
an Item object using the initial amount paramAmount, and then checks that isInStock()
returns a value of paramExpectedResult.
Test Suites
A common approach in testing is to gather a series of tests into a suite. For example, imagine we had a series of tests
for a VendingMachine class (omitted)
in addition to our Item tests (the idea being that the VendingMachine contains several Items):
import static org.junit.Assert.*;
import org.junit.Test;
public class VendingMachineTest {
@Test
public void testAddItem()
{
VendingMachine vm=new VendingMachine();
vm.addItem(id,new Item("Creme Egg",0.60,101));
assertTrue(vm.itemIsPresent(101));
}
@Test
public void testSellOK()
{
VendingMachine vm = new VendingMachine();
vm.addItem(101,new Item("Creme Egg",0.60,101));
vm.insertMoney(0.60);
assertTrue(vm.sell(101));
}
@Test
public void testSellInsufficientMoney()
{
VendingMachine vm = new VendingMachine();
vm.addItem(101,new Item("Creme Egg",0.60,101));
vm.insertMoney(0.59);
assertFalse(vm.sell(101));
}
}
(These tests are used to test whether an item was successfully added, whether an item can be sold if
enough money was inserted, and whether an item is unable to be sold if not enough money was
inserted. Note also here the use of assertTrue() and assertFalse() which are shorthand versions of
assertEquals() in cases where we know the method will return true or false).
References