Code Testing

Code Testing#

Code testing involves the process of writing formal tests that ensure your code is working as intended. In this chapter, we’ll focus on the most common type of code test - the unit test - discussing how to write basic tests with assert, what makes a good test, and how to write and run an organized suite of tests using Python’s built-in unittest module.

What is a Unit Test?#

A unit test checks that one small, isolated unit of your code (usually a single function) behaves the way you expect it to.

Key ideas:

Tests are code that calls your code and checks the output
If the output matches what you expected, the test passes
If the output does not match, the test fails

A helpful analogy is the taste test: you follow the recipe (your function), then taste the result (your test) to see if it came out right.

A unit test is code that checks that a single, isolated piece of your code (typically a function) behaves as expected.

Why Test?#

Catch bugs early - before they wreak havoc elsewhere in your code
Understand your own code - writing tests forces you to think carefully about what your function should do
Refactor safely - if you change your code later, your tests will tell you immediately whether you’ve broken something
Professional practice - out in the world, code that matters is code that is tested

Writing Basic Tests with `assert`#

The simplest way to test in Python is the assert statement, which you’ve seen throughout the Practice sections of this book.

Syntax:

assert <expression>, "Optional error message if this fails"

If <expression> is True, nothing happens (the test “passes silently”)
If <expression> is False, Python raises an AssertionError

# a passing assert - nothing is printed, no error raised
assert 2 + 2 == 4

# a failing assert - raises an AssertionError
assert 2 + 2 == 5

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[2], line 2
      1 # a failing assert - raises an AssertionError
----> 2 assert 2 + 2 == 5

AssertionError: 

# a failing assert with a helpful message
assert 2 + 2 == 5, "2 + 2 does not equal 5!"

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[3], line 2
      1 # a failing assert with a helpful message
----> 2 assert 2 + 2 == 5, "2 + 2 does not equal 5!"

AssertionError: 2 + 2 does not equal 5!

Testing a Function with `assert`#

To test a function with assert, we call the function with inputs where we know what the output should be, and assert that the function returns that value. (Note: this is what the Practice sections of this book have been doing all along! However, this is not yet a unit test…we’re getting there.)

# define a function we want to test
def add(a, b):
    """Return the sum of a and b."""
    return a + b

# test: two positive integers
assert add(2, 3) == 5

# test: a negative number
assert add(-1, 1) == 0

# test: floats
assert add(0.1, 0.2) == 0.30000000000000004   # floating-point quirk!

# test: zero
assert add(0, 0) == 0

print("All tests passed!")

All tests passed!

Notice the float test above - floating-point arithmetic in Python does not behave the way you might expect from math class. Testing helps you discover these surprises.

0.1 + 0.2  # returns 0.30000000000000004  (not 0.3!)

# convince ourselves of float weirdness
print(0.1 + 0.2)
print(0.1 + 0.2 == 0.3)

0.30000000000000004
False

What Makes a Good Unit Test?#

Not all tests are created equal. A great test suite is just as important as great code.

Here are the key properties of a good unit test:

1. Tests exactly ONE thing#

Each test should have a single, clear purpose. If a test fails, you should immediately know what broke.

Avoid - one test checking many unrelated things:

# to be avoided: checking multiple unrelated behaviors in one test
# if this fails, which part failed?
def test_everything():
    assert add(2, 3) == 5
    assert add(-1, 1) == 0
    assert type(add(1, 2)) == int
    assert len("hello") == 5       # doesn't even test add()!
    assert add(100, 200) == 300

Prefer - each test has one job:

def test_add_positive_numbers():
    assert add(2, 3) == 5

def test_add_negative_numbers():
    assert add(-1, -2) == -3

def test_add_returns_correct_type():
    assert type(add(1, 2)) == int

test_add_positive_numbers()
test_add_negative_numbers()
test_add_returns_correct_type()

2. Has a descriptive name#

The test name should read like a sentence describing what behavior is being checked. When a test fails, its name is the first thing you read.

Avoid	Prefer
`test_1()`	`test_add_returns_zero_when_inputs_cancel()`
`test_func()`	`test_is_palindrome_ignores_spaces()`
`my_test()`	`test_count_vowels_on_empty_string()`

3. Covers edge cases#

Most bugs hide in edge cases - the unusual inputs you didn’t think about when writing the function.

Common edge cases to always think about:

Empty input: empty string "", empty list [], zero 0
Negative numbers: does your function handle them?
One element: a list with a single item
Large values: what happens with a very big number?
Wrong type: what if someone passes a number where a string is expected?

def count_words(sentence):
    """Return the number of words in a sentence."""
    return len(sentence.split())

# normal case
assert count_words("hello world") == 2

# edge case: single word
assert count_words("hello") == 1

# edge case: empty string
assert count_words("") == 0

# edge case: space at end of word
assert count_words("hello ") == 1

# edge case: space at beginning of word
assert count_words(" hello") == 1

# edge case: lots of spaces between words
assert count_words("hello   world") == 2   # .split() handles this correctly!

# edge case: punctuation included
assert count_words("hello world!") == 2

print("All edge case tests passed!")

All edge case tests passed!

4. Is independent#

Tests should NOT depend on each other.

Running them in a different order should give the same result
Output from one test should not be needed for another

Testing with `unittest`#

Writing raw assert statements works, but Python’s built-in unittest module gives us a much more organized, scalable way to write tests.

With unittest:

Tests are grouped into classes that inherit from unittest.TestCase
The class name starts with Test…and then includes what it’s testing
Each test is a method that starts with test_
You get helpful assertion methods (more readable than plain assert)
You get a clear summary of which tests passed and which failed

Basic Structure#

import unittest

# step 1: define a class that inherits from unittest.TestCase
class TestAdd(unittest.TestCase):

    # step 2: each test is a method starting with test_
    def test_add_two_positives(self):
        self.assertEqual(add(2, 3), 5)

    def test_add_negative_and_positive(self):
        self.assertEqual(add(-1, 1), 0)

    def test_add_two_negatives(self):
        self.assertEqual(add(-3, -7), -10)

    def test_add_zeros(self):
        self.assertEqual(add(0, 0), 0)

Running `unittest` inside a Jupyter Notebook#

Tests are typically run from an external file (coming up later in this chapter!), but you can run a test class defined in a notebook, as we see here. (The class name in argv specifies which test class to run; the semicolon suppresses an extra line of output. And, if you were to add teh argument verbosity=2, it would give detailed output for each test.)

# run the test class defined above
unittest.main(argv=['', 'TestAdd'], exit=False);

....
----------------------------------------------------------------------
Ran 4 tests in 0.010s

OK

Reading the output:

ok = test passed
FAIL = test ran but the assertion was wrong
ERROR = test crashed with an unexpected exception

`unittest` Assertion Methods#

Instead of raw assert, unittest.TestCase gives you purpose-built methods. These produce much better error messages when a test fails.

Method	What it checks
`self.assertEqual(a, b)`	`a == b`
`self.assertNotEqual(a, b)`	`a != b`
`self.assertTrue(x)`	`bool(x) is True`
`self.assertFalse(x)`	`bool(x) is False`
`self.assertIsNone(x)`	`x is None`
`self.assertIn(a, b)`	`a in b`
`self.assertIsInstance(a, b)`	`isinstance(a, b)`
`self.assertRaises(Error, func, args)`	`func(args)` raises `Error`
`self.assertAlmostEqual(a, b)`	`a` is approximately equal to `b` (useful for floats!)

def divide(num1, num2):
    return num1 / num2

# examples of each of the above
class TestAssertionMethods(unittest.TestCase):

    def test_equal(self):
        self.assertEqual(1 + 1, 2)

    def test_not_equal(self):
        self.assertNotEqual("hello", "world")

    def test_true(self):
        self.assertTrue(5 > 3)

    def test_false(self):
        self.assertFalse(3 > 5)

    def test_in(self):
        self.assertIn("cat", ["dog", "cat", "fish"])

    def test_isinstance(self):
        self.assertIsInstance(divide(10, 5), float)

    def test_almost_equal(self):
        # perfect for floats - checks within 7 decimal places by default
        self.assertAlmostEqual(0.1 + 0.2, 0.3)

    def test_raises(self):
        # check that divide() raises ZeroDivisionError when num2 is 0
        self.assertRaises(ZeroDivisionError, divide, 10, 0)

unittest.main(argv=['', 'TestAssertionMethods'], exit=False);

........
----------------------------------------------------------------------
Ran 8 tests in 0.015s

OK

`setUp` and `tearDown`#

If multiple tests need the same setup (e.g., creating an object to test), use setUp(). It runs automatically before each test method.

tearDown() runs after each test and is useful for cleanup.

class ShoppingCart:
    """A very simple shopping cart."""

    def __init__(self):
        self.items = []

    def add_item(self, item):
        self.items.append(item)

    def remove_item(self, item):
        self.items.remove(item)

    def total_items(self):
        return len(self.items)

    def is_empty(self):
        return len(self.items) == 0

class TestShoppingCart(unittest.TestCase):

    def setUp(self):
        # this runs before EVERY test method below
        # each test gets a fresh cart - they can't interfere with each other!
        self.cart = ShoppingCart()

    def test_new_cart_is_empty(self):
        self.assertTrue(self.cart.is_empty())

    def test_add_item_increases_count(self):
        self.cart.add_item("apple")
        self.assertEqual(self.cart.total_items(), 1)

    def test_add_multiple_items(self):
        self.cart.add_item("apple")
        self.cart.add_item("banana")
        self.assertEqual(self.cart.total_items(), 2)

    def test_remove_item_decreases_count(self):
        self.cart.add_item("apple")
        self.cart.remove_item("apple")
        self.assertTrue(self.cart.is_empty())

    def test_cart_contains_added_item(self):
        self.cart.add_item("mango")
        self.assertIn("mango", self.cart.items)

unittest.main(argv=['', 'TestShoppingCart'], exit=False);

.....
----------------------------------------------------------------------
Ran 5 tests in 0.008s

OK

Notice: because setUp creates a fresh self.cart before each test, none of these tests interfere with each other - even though they all share the same class.

Using Tests to Check a Function’s Behavior#

Tests aren’t just for checking outputs - they can also check that a function fails the way it’s supposed to. Consider the function below. Its docstring promises that it raises an IndexError on an empty list. Does it?

def get_last_element(lst):
    """Return the last element of a list.

    Should raise IndexError on an empty list.
    """
    return lst[-1]

get_last_element([1, 2, 3])

get_last_element([])

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In[20], line 1
----> 1 get_last_element([])

Cell In[18], line 6, in get_last_element(lst)
      2     """Return the last element of a list.
      3 
      4     Should raise IndexError on an empty list.
      5     """
----> 6     return lst[-1]

IndexError: list index out of range

It does - but rather than crashing our notebook to find out, we can capture that expectation in a test, using assertRaises. Note that there are two ways to use assertRaises: passing the function and its arguments directly, or using a with statement (helpful when the code you’re testing is more than a single function call).

class TestGetLastElement(unittest.TestCase):

    def test_output_integers(self):
        self.assertEqual(get_last_element([1, 2, 3]), 3)

    def test_output_strings(self):
        self.assertEqual(get_last_element(['a', 'b', 'c', 'd']), 'd')

    def test_empty_list_raises_error(self):
        self.assertRaises(IndexError, get_last_element, [])

    def test_empty_list_raises_error_with_statement(self):
        with self.assertRaises(IndexError):
            get_last_element([])

unittest.main(argv=['', 'TestGetLastElement'], exit=False);

....
----------------------------------------------------------------------
Ran 4 tests in 0.004s

OK

What Happens When a Test Fails?#

Let’s intentionally write a broken test to see what the failure output looks like:

class TestIntentionalFailure(unittest.TestCase):

    def test_this_will_fail(self):
        self.assertEqual(get_last_element([1, 2, 3]), 1)

unittest.main(argv=['', 'TestIntentionalFailure'], exit=False);

F
======================================================================
FAIL: test_this_will_fail (__main__.TestIntentionalFailure.test_this_will_fail)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/folders/jp/hdbfltdj035719571j9wynwm0000gn/T/ipykernel_46127/3836681237.py", line 4, in test_this_will_fail
    self.assertEqual(get_last_element([1, 2, 3]), 1)
AssertionError: 3 != 1

----------------------------------------------------------------------
Ran 1 test in 0.006s

FAILED (failures=1)

The failure output tells you:

Which test failed: test_this_will_fail
What the values were: 3 != 1'
Where in the code: the line number of the failing assertion

This is why using self.assertEqual (instead of raw assert) is so helpful - the error message is much more informative.

Summary#

Concept	Key Takeaway
Unit test	Code that checks one function does what it’s supposed to
`assert`	The simplest way to test; raises `AssertionError` if `False`
Good test	Tests one thing, has a clear name, covers edge cases, is independent
`unittest.TestCase`	Organized, class-based testing with helpful assertion methods
`setUp()`	Runs before each test; use it to avoid repeating setup code

Exercises#

Q1. Which of the following is NOT a property of a good unit test?

A) It tests exactly one thing
B) It has a descriptive name
C) It covers edge cases
D) It depends on the output of the test that runs before it
E) It is independent of other tests

Q2. In unittest output, a test that crashes with an unexpected exception (rather than failing an assertion) is reported as:

A) ok
B) FAIL
C) ERROR
D) skipped

Q3. Why does self.assertEqual(0.1 + 0.2, 0.3) fail, and which assertion method should be used instead?

Q4. When does the setUp() method run?

A) Once, before all of the test methods in the class
B) Before each test method in the class
C) Only when a test fails
D) Once, after all of the test methods in the class

Q5. The function absolute_value is provided below. Write a test class TestAbsoluteValue with at least three test methods: one testing a positive input, one testing a negative input, and one testing the edge case 0. Then, run your tests.

def absolute_value(number):
    """Return the absolute value of the input number."""
    if number < 0:
        return -number
    return number

Q6. Using the divide function defined earlier in this chapter, write a test class TestDivide with two test methods: one checking that divide(10, 5) returns the expected value and one checking that dividing by zero raises a ZeroDivisionError.

Answers to Exercises

Q1. D — tests should be independent: running them in a different order should give the same result, and no test should rely on another test’s output.

Q2. C — ERROR means the test crashed with an unexpected exception before its assertion could be evaluated. FAIL means the test ran, but the assertion was wrong.

Q3. Because of floating-point arithmetic, 0.1 + 0.2 evaluates to 0.30000000000000004, which is not exactly equal to 0.3, so assertEqual fails. Use self.assertAlmostEqual(0.1 + 0.2, 0.3) instead — it checks that the two values are equal to within 7 decimal places (by default), which is what you almost always want for floats.

Q4. B — setUp() runs before each test method, giving every test a fresh setup so tests can’t interfere with one another.

Q5. For example:

class TestAbsoluteValue(unittest.TestCase):

    def test_positive_input(self):
        self.assertEqual(absolute_value(5), 5)

    def test_negative_input(self):
        self.assertEqual(absolute_value(-5), 5)

    def test_zero(self):
        self.assertEqual(absolute_value(0), 0)

unittest.main(argv=['', 'TestAbsoluteValue'], verbosity=2, exit=False);

Q6. For example:

class TestDivide(unittest.TestCase):

    def test_expected_output(self):
        self.assertEqual(divide(10, 5), 2.0)

    def test_divide_by_zero_raises(self):
        self.assertRaises(ZeroDivisionError, divide, 10, 0)

unittest.main(argv=['', 'TestDivide'], verbosity=2, exit=False);

(For the second test, with self.assertRaises(ZeroDivisionError): followed by divide(10, 0) on the next line works as well.)