Code Testing

Code Testing#

Code testing involves the process of writing formal tests that ensure your code is working as intended. These tests can be compiled into a formal testing suite. The process of including a testing suite within a coding project not only enables you, the programmer, to efficiently check that your code is working as intended, but it also demonstrates to others looking through your projects that your code has been tested, making it more trustworthy.

Types of Code Testing#

While there are more than the tests described here, these four are core to understanding testing code projects early on in one’s programming journey. In fact, if you’ve been writing bits of code to ensure that the function or class you’ve just written is working as intended, you’ve already been writing the first kind of test, a smoke test. Smoke tests are not formal tests, but rather they are the preliminary checks you run when writing your code to ensure that it’s working as intended. Unit tests, on the other hand, are formal tests that ensure the code you’ve written is behaving as expected. These are the tests we’ll focus on in this chapter. There are a number of different platforms that exist for code testing in python; however, here we’ll focus on the unittest framework to implement unit testing. Beyond unit testing, integration and sytem tests go beyond testing a single function or class to testing an entire code project. While we won’t go further with these here, it’s important to know that there is testing beyond unit tests.

Smoke tests: preliminary tests to check basic functionality (gut check)
Unit tests (primary focus): test functions & objects to ensure that the code is behaving as expected
Integration tests: test functions, classes & modules interacting
System tests: tests end-to-end behavior

Test-Driven Development#

In software development, test-driven development is an approach in which you write tests first - and then write code to pass the tests.

Ensures you go into writing code with a good plan / outline
Ensures that you have a test suite, as you can not decide to neglect to test your code after the fact you made the test suite
Note: when you complete (or at least write) assignments for this class, you are effectively doing test-driven development

Unit Tests#

When we consider writing unit tests using the unittest framework, we’ll consider the following:

We write one test for each “piece” of your code (function, class, etc.)
Tests should ensure that the code is operating as expected
Testing should consider “edge (atypical) cases”
unittest will let you know which tests pass/fail/throw an error

By formalizing tests using the unittest framework will help you resist the urge to assume computers will act how you think it will work.

How to Write Tests#

Given a function or class you want to test:

You need to have an expectation for what it should do
Write out some example cases, with known answers
Use assert to check that your example cases do run as expected
Collect these examples into test functions, stored in test files

`unittest`#

When introducing the unittest framework, it’s important to know that its functionality will not be available until it’s imported:

import unittest

`unittest` template#

After importing, below you’ll find the basic unittest framework. Notably:

the class name must start with Test
After Test the class name typically indicates the “piece” of code (function, class, etc.) being tested.
Within the parentheses we inherit TestCase from unittest to enable use of its testing statements
The methods within the class must start with test_
The rest of the method name after test_ typically refer to what about the piece of code you’re testing
Within each method, you can have any number of assert statements.

We’ll review a number of different assert statements and the cases in which you would use each in this chapter. The last three lines below run the test TestName. As you see, for each test/method within the class, we get a readout as to whether the test passes. Note taht test_true fails because what is within the parentheses (False) is NOT a true statement. This is included so you see what a failing test would look like; however, when we write tests, we’re writing them with the expectation/aim that they will all pass

class TestName(unittest.TestCase):

    def test_equal(self):
        self.assertEqual(1, 1)

    def test_true(self):
        self.assertTrue(False)

    # ...

if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(TestName)
    unittest.TextTestRunner(verbosity=2).run(suite)

test_equal (__main__.TestName.test_equal) ... ok
test_true (__main__.TestName.test_true) ... FAIL

======================================================================
FAIL: test_true (__main__.TestName.test_true)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/folders/jp/hdbfltdj035719571j9wynwm0000gn/T/ipykernel_22759/3539443511.py", line 7, in test_true
    self.assertTrue(False)
AssertionError: False is not true

----------------------------------------------------------------------
Ran 2 tests in 0.026s

FAILED (failures=1)

`assert` statements#

While there are many different assert statemetns within unittest, we’ll introduce a few of the most commonly-used here.

`.assertEqual(a, b)`#

Checks if a is equal to b. Similar to the expression a == b.

Note: Although it doesn’t affect the results, it’s good practice to let a be the expected value (what you think the result will be) and b be the actual value (what the result actually is).

For example, if we wanted to check the add provided here:

def add(a, b):
    return a + b

…we would want to consider our expectations of this function. Specifically, what would we expect the output from this function to be? Well, if we gave the inputs 4 and 6, we’d expect the output of the function to be 10. A perfect use case for assertEqual:

# Equality testing suite
class TestAdd(unittest.TestCase):

    def test_outcome(self):
        self.assertEqual(10, add(4, 6))

if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(TestAdd)
    unittest.TextTestRunner(verbosity=2).run(suite)

test_outcome (__main__.TestAdd.test_outcome) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.001s

OK

`.assertTrue(x)`#

Any time you have an expression that you expect to evaluate as True, you can use assertTrue, which checks if x is True. Same as the expression bool(x) == True.

# Truth testing suite
class TestTrue(unittest.TestCase):

    def test_true(self):
        self.assertTrue(add(4, 6) == 10)

if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(TestTrue)
    unittest.TextTestRunner(verbosity=2).run(suite)

test_true (__main__.TestTrue.test_true) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.001s

OK

`.assertFalse(x)`#

On the other hand, assertFalse checks if x is False. Same as the expression bool(x) == False.

# False testing suite
class TestFalse(unittest.TestCase):

    def test_false(self):
        self.assertFalse(add(4, 6) == 11)

if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(TestFalse)
    unittest.TextTestRunner(verbosity=2).run(suite)

test_false (__main__.TestFalse.test_false) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.001s

OK

`.assertIsInstance(a, b)`#

assertIsInstance checks if a is of variable type b (int, float, str, bool) or is of an instance of class b. Similar to the expression isinstance(a, b) or type(a) == b.

# Instance testing suite
class TestInstance(unittest.TestCase):
    
    def test_instance(self):
        self.assertIsInstance(add(4, 6), int)

if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(TestInstance)
    unittest.TextTestRunner(verbosity=2).run(suite)

test_instance (__main__.TestInstance.test_instance) ... ok

----------------------------------------------------------------------
Ran 1 test in 0.001s

OK

`.assertIn(a, b)`#

Finally, assertIn checks for membership, if a belongs in b. Same as the expression a in b. Here, rather than test our add function, we’ll demonstrate that you can define an attribute (here fruits) and then test it within each of the methods:

# Membership testing suite
class TestIn(unittest.TestCase):

    fruits = ['apples', 'bananas', 'oranges']

    def test_membership(self):
        self.assertIn('apples', self.fruits)

    def test_membership_bad(self):
        self.assertIn('pears', self.fruits)

if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(TestIn)
    unittest.TextTestRunner(verbosity=2).run(suite)

test_membership (__main__.TestIn.test_membership) ... ok
test_membership_bad (__main__.TestIn.test_membership_bad) ... FAIL

======================================================================
FAIL: test_membership_bad (__main__.TestIn.test_membership_bad)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/folders/jp/hdbfltdj035719571j9wynwm0000gn/T/ipykernel_22759/4249137848.py", line 10, in test_membership_bad
    self.assertIn('pears', self.fruits)
AssertionError: 'pears' not found in ['apples', 'bananas', 'oranges']

----------------------------------------------------------------------
Ran 2 tests in 0.002s

FAILED (failures=1)

A unit test#

Putting the above all together, we could test our add function with a few different assert statements, all of which we would expect to be be true if our function is working as intended. Specifically, test_instance checks that the output given two different inputs would be of the expected type. test_output checks that the specific values returned are correct. We’ve even snuck in an additional assert statement - assertAlmostEqual, which is helpful when checking float values!

class TestAllAdd(unittest.TestCase):
   
    def test_instance(self):
        self.assertIsInstance(add(4, 6), int)
        self.assertIsInstance(add(4.5, 6), float)

    def test_output(self):
        self.assertEqual(10, add(4, 6))
        self.assertEqual(4, add(-2, 6))
        self.assertAlmostEqual(10.5, add(4.5, 6))

if __name__ == '__main__':
    suite = unittest.TestLoader().loadTestsFromTestCase(TestAllAdd)
    unittest.TextTestRunner(verbosity=2).run(suite)

test_instance (__main__.TestAllAdd.test_instance) ... ok
test_output (__main__.TestAllAdd.test_output) ... ok

----------------------------------------------------------------------
Ran 2 tests in 0.001s

OK

Code Testing

Contents

Code Testing#

Types of Code Testing#

Test-Driven Development#

Unit Tests#

How to Write Tests#

unittest#

unittest template#

assert statements#

.assertEqual(a, b)#

.assertTrue(x)#

.assertFalse(x)#

.assertIsInstance(a, b)#

.assertIn(a, b)#