Lecture 2c: More details of TDD

Handling odd behaviors

To explore another feature of pytest, we’ll consider another aspect of our number_negatives() function. Specifically, what should we do if an invalid sequence is entered? A sensible thing to do in this case is to make our software throw a RuntimeError.

Again, in designing our test, we need to think about what constitutes an invalid sequence. We’ll only allow the 20 canonical symbols for residues. So, we adjust our test function accordingly. We cannot use the assert statement to check for proper error handling, so we use the pytest.raises() function. This function takes as its first argument the type of exception expected, and a string containing the code to be run to give the exception.

A note on assertions vs raising exceptions

It is important to draw the distinction between assertions and raising exceptions in our code.

* We should raise exceptions when we are checking inputs to our function. I.e., we are checking to make sure the user is using the function properly. * We should use assertions to make sure the function operates as expected for given input.

We should then add to the code of the test_seq_features.py to include our expectation that the program should throw a RuntimeError if an invalid sequence is entered:

def test_number_negatives_for_invalid_amino_acid():
    with pytest.raises(RuntimeError) as excinfo:
        seq_features.number_negatives('Z')
    excinfo.match("Z is not a valid amino acid")

We also have to include import pytest at the beginning of the test_seq_features.py file because we are using the pytest.raises() function. It is clear that if Z is passed as the input sequence, the program should throw a RuntimeError saying: “Z is an invalid sequence.” Let’s test.

!pytest -v

============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-5.1.2, py-1.8.0, pluggy-0.13.0 -- /Users/Justin/anaconda3/bin/python
cachedir: .pytest_cache
rootdir: /Users/Justin/Dropbox/git/bebi103_course
collected 5 items

test_seq_features.py::test_number_negatives_single_E_or_D PASSED         [ 20%]
test_seq_features.py::test_number_negatives_for_empty PASSED             [ 40%]
test_seq_features.py::test_number_negatives_for_short_sequences PASSED   [ 60%]
test_seq_features.py::test_number_negatives_for_lowercase PASSED         [ 80%]
test_seq_features.py::test_number_negatives_for_invalid_amino_acid FAILED [100%]

=================================== FAILURES ===================================
_________________ test_number_negatives_for_invalid_amino_acid _________________

    def test_number_negatives_for_invalid_amino_acid():
        with pytest.raises(RuntimeError) as excinfo:
>           seq_features.number_negatives('Z')
E           Failed: DID NOT RAISE <class 'RuntimeError'>

test_seq_features.py:29: Failed
========================= 1 failed, 4 passed in 0.07s ==========================

Although all other four tests still pass, the last one fails because our program does not know yet to throw a RuntimeError when it receives an invalid sequence as input. Let’s fix that. We adjust the function in the seq_features.py file to be as follows. (More detailed discussion on exceptions and error handling in in lesson 2.)

def number_negatives(seq):
    """Number of negative residues a protein sequence"""
    # Convert sequence to upper case
    seq = seq.upper()

    if seq == 'Z':
        raise RuntimeError('Z is not a valid amino acid.')

    # Count E's and D's, since these are the negative residues
    return seq.count('E') + seq.count('D')

Now, re-running the test…

!pytest -v

============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-5.1.2, py-1.8.0, pluggy-0.13.0 -- /Users/Justin/anaconda3/bin/python
cachedir: .pytest_cache
rootdir: /Users/Justin/Dropbox/git/bebi103_course
collected 5 items

test_seq_features.py::test_number_negatives_single_E_or_D PASSED         [ 20%]
test_seq_features.py::test_number_negatives_for_empty PASSED             [ 40%]
test_seq_features.py::test_number_negatives_for_short_sequences PASSED   [ 60%]
test_seq_features.py::test_number_negatives_for_lowercase PASSED         [ 80%]
test_seq_features.py::test_number_negatives_for_invalid_amino_acid PASSED [100%]

============================== 5 passed in 0.02s ===============================

Obviously, this is not a very robust fix; it only works if the invalid amino acid is Z. We need a smarter way to fix this. We can adjust the contents of your seq_features.py file as follows.

def number_negatives(seq):
    """Number of negative residues a protein sequence"""
    # Convert sequence to upper case
    seq = seq.upper()

    valid_aas = "ARNDCQEGHILKMFPSTWYV"

    # Check for a valid sequence
    for aa in seq:
        if aa not in valid_aas:
            raise RuntimeError(aa + ' is not a valid amino acid.')

    # Count E's and D's, since these are the negative residues
    return seq.count('E') + seq.count('D')

Now let’s run pytest one more time.

!pytest -v

============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-5.1.2, py-1.8.0, pluggy-0.13.0 -- /Users/Justin/anaconda3/bin/python
cachedir: .pytest_cache
rootdir: /Users/Justin/Dropbox/git/bebi103_course
collected 5 items

test_seq_features.py::test_number_negatives_single_E_or_D PASSED         [ 20%]
test_seq_features.py::test_number_negatives_for_empty PASSED             [ 40%]
test_seq_features.py::test_number_negatives_for_short_sequences PASSED   [ 60%]
test_seq_features.py::test_number_negatives_for_lowercase PASSED         [ 80%]
test_seq_features.py::test_number_negatives_for_invalid_amino_acid PASSED [100%]

============================== 5 passed in 0.02s ===============================

All of our tests passed!

Summary of TDD

Now that you have some experience with TDD and have an idea about what it is and how it works, let’s formalize things by writing out the basic principles of test-driven development.

1. Build your software out of small functions that do one specific thing.

2. Build unit tests for all of your functions.

3. Whenever you want to make any enhancements of adjustments to your code, write tests for it first.

4. Whenever you encounter a bug, write tests for it that reproduce the behavior and then fix the code to make the entire test suite to pass.

Improving the seq_features module using TDD: Practice

Let’s write now a function that will calculate the total number of positively charged residues in a protein. In other words, let’s count the number of Lysine (K), Arginine (R) and Histidine (H) residues in the sequence.

To do that, let’s make the prototype function and add to seq_features.py:

def number_positives(seq):
    """Number of positive residues a protein sequence"""
    pass

and now, let’s build a simple test and add it to test_seq_features.py

def test_number_positives_single_R_K_or_H():
    """Perform unit tests on number_positives for single AA"""
    assert seq_features.number_positives('R') == 1
    assert seq_features.number_positives('K') == 1
    assert seq_features.number_positives('H') == 1

and let’s test. We will use the -W ignore::DeprecationWarning so pytest does not spit all of the deprecation warnings to the screen.

!pytest -v -W ignore::DeprecationWarning

============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-5.1.2, py-1.8.0, pluggy-0.13.0 -- /Users/Justin/anaconda3/bin/python
cachedir: .pytest_cache
rootdir: /Users/Justin/Dropbox/git/bebi103_course
collected 6 items

test_seq_features.py::test_number_negatives_single_E_or_D PASSED         [ 16%]
test_seq_features.py::test_number_negatives_for_empty PASSED             [ 33%]
test_seq_features.py::test_number_negatives_for_short_sequences PASSED   [ 50%]
test_seq_features.py::test_number_negatives_for_lowercase PASSED         [ 66%]
test_seq_features.py::test_number_negatives_for_invalid_amino_acid PASSED [ 83%]
test_seq_features.py::test_number_positives_single_R_K_or_H FAILED       [100%]

=================================== FAILURES ===================================
____________________ test_number_positives_single_R_K_or_H _____________________

    def test_number_positives_single_R_K_or_H():
        """Perform unit tests on number_positives for single AA"""
>       assert seq_features.number_positives('R') == 1
E       assert None == 1
E         -None
E         +1

test_seq_features.py:35: AssertionError
========================= 1 failed, 5 passed in 0.07s ==========================

Let’s fix our function, which failed by design.

def number_positives(seq):
    """Number of positive residues a protein sequence"""
    # Count R's, K's and H's, since these are the positive residues
    return seq.count('R') + seq.count('K') + seq.count('H')

And test again…

!pytest -v -W ignore::DeprecationWarning

============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-5.1.2, py-1.8.0, pluggy-0.13.0 -- /Users/Justin/anaconda3/bin/python
cachedir: .pytest_cache
rootdir: /Users/Justin/Dropbox/git/bebi103_course
collected 6 items

test_seq_features.py::test_number_negatives_single_E_or_D PASSED         [ 16%]
test_seq_features.py::test_number_negatives_for_empty PASSED             [ 33%]
test_seq_features.py::test_number_negatives_for_short_sequences PASSED   [ 50%]
test_seq_features.py::test_number_negatives_for_lowercase PASSED         [ 66%]
test_seq_features.py::test_number_negatives_for_invalid_amino_acid PASSED [ 83%]
test_seq_features.py::test_number_positives_single_R_K_or_H PASSED       [100%]

============================== 6 passed in 0.02s ===============================

Now, obviously we want the number_positives() function to behave like the number_negatives() with weird cases, let’s add the tests below to test_seq_features.py.

def test_number_positives_for_empty():
    """Perform unit tests on number_positives for empty entry"""
    assert seq_features.number_positives('') == 0


def test_number_positives_for_short_sequences():
    """Perform unit tests on number_positives for short sequence"""
    assert seq_features.number_positives('RCKLWTTRE') == 3
    assert seq_features.number_positives('DDDDEEEE') == 0


def test_number_positives_for_lowercase():
    """Perform unit tests on number_positives for lowercase"""
    assert seq_features.number_positives('rcklwttre') == 3


def test_number_positives_for_invalid_amino_acid():
    with pytest.raises(RuntimeError) as excinfo:
        seq_features.number_positives('Z')
    excinfo.match("Z is not a valid amino acid")

Let’s test it.

!pytest -v -W ignore::DeprecationWarning

============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-5.1.2, py-1.8.0, pluggy-0.13.0 -- /Users/Justin/anaconda3/bin/python
cachedir: .pytest_cache
rootdir: /Users/Justin/Dropbox/git/bebi103_course
collected 10 items

test_seq_features.py::test_number_negatives_single_E_or_D PASSED         [ 10%]
test_seq_features.py::test_number_negatives_for_empty PASSED             [ 20%]
test_seq_features.py::test_number_negatives_for_short_sequences PASSED   [ 30%]
test_seq_features.py::test_number_negatives_for_lowercase PASSED         [ 40%]
test_seq_features.py::test_number_negatives_for_invalid_amino_acid PASSED [ 50%]
test_seq_features.py::test_number_positives_single_R_K_or_H PASSED       [ 60%]
test_seq_features.py::test_number_positives_for_empty PASSED             [ 70%]
test_seq_features.py::test_number_positives_for_short_sequences PASSED   [ 80%]
test_seq_features.py::test_number_positives_for_lowercase FAILED         [ 90%]
test_seq_features.py::test_number_positives_for_invalid_amino_acid FAILED [100%]

=================================== FAILURES ===================================
_____________________ test_number_positives_for_lowercase ______________________

    def test_number_positives_for_lowercase():
        """Perform unit tests on number_positives for lowercase"""
>       assert seq_features.number_positives('rcklwttre') == 3
E       assert 0 == 3
E         -0
E         +3

test_seq_features.py:53: AssertionError
_________________ test_number_positives_for_invalid_amino_acid _________________

    def test_number_positives_for_invalid_amino_acid():
        with pytest.raises(RuntimeError) as excinfo:
>           seq_features.number_positives('Z')
E           Failed: DID NOT RAISE <class 'RuntimeError'>

test_seq_features.py:58: Failed
========================= 2 failed, 8 passed in 0.09s ==========================

Although the current version of the function number_positives() passes most of the tests, it is not ready to handle to the edge cases (lowercases and invalid amino-acids).

We can fix that easily; let’s update the number_positives()…

def number_positives(seq):
    """Number of positive residues a protein sequence"""
    # Convert sequence to upper case
    seq = seq.upper()

    # Check for a valid sequence
    for aa in seq:
        if aa not in bootcamp_utils.aa.keys():
            raise RuntimeError(aa + ' is not a valid amino acid.')

    return seq.count('R') + seq.count('K') + seq.count('H')

…and run the test one more time:

!pytest -v -W ignore::DeprecationWarning

============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-5.1.2, py-1.8.0, pluggy-0.13.0 -- /Users/Justin/anaconda3/bin/python
cachedir: .pytest_cache
rootdir: /Users/Justin/Dropbox/git/bebi103_course
collected 10 items

test_seq_features.py::test_number_negatives_single_E_or_D PASSED         [ 10%]
test_seq_features.py::test_number_negatives_for_empty PASSED             [ 20%]
test_seq_features.py::test_number_negatives_for_short_sequences PASSED   [ 30%]
test_seq_features.py::test_number_negatives_for_lowercase PASSED         [ 40%]
test_seq_features.py::test_number_negatives_for_invalid_amino_acid PASSED [ 50%]
test_seq_features.py::test_number_positives_single_R_K_or_H PASSED       [ 60%]
test_seq_features.py::test_number_positives_for_empty PASSED             [ 70%]
test_seq_features.py::test_number_positives_for_short_sequences PASSED   [ 80%]
test_seq_features.py::test_number_positives_for_lowercase PASSED         [ 90%]
test_seq_features.py::test_number_positives_for_invalid_amino_acid PASSED [100%]

============================== 10 passed in 0.04s ==============================

We now have a good set of tests and functions that work as expected as a result.

Code refactoring and TDD

As we are building modules and functions, though we may try, we are not able to anticipate all the functionalities they must have. And by adding new functionalities, we might need to change our code substantially and even dramatically change the initial logic that worked so well up to this point. This is so common in programming that developers have a name for it: code refactoring.

For example, we did not anticipate when we start writing seq_features that we also wanted to calculate the positive charges as well. Beyond that, we broke one of the most important rules in programming: functions must do one thing and only one thing very well. It is clear that number_negatives() was doing three things:

1. Dealing with lowercases characters.

2. Raising exceptions for invalid amino-acids in the input sequence.

3. Calculating the negative charge of amino-acids.

Turns out that number_positives() also needs to do items 1 and 2, and because of that we have repeated the following lines of code in two different functions, within the same module:

# Convert sequence to upper case
 seq = seq.upper()

 valid_aas = "ARNDCQEGHILKMFPSTWYV"

 # Check for a valid sequence
 for aa in seq:
     if aa not in valid_aas:
         raise RuntimeError(aa + ' is not a valid amino acid.')

and if we are trying to make this module more robust, every time we catch a bug, we will need to change identical code in two places. So let’s perform a code refactoring in order to keep the principle of functions doing only one thing as close to the truth as possible.

The first task, changing the inputted sequence to uppercase, uses a built-in Python function, and using another function to do this is unnecessary. So, we can keep the seq = seq.upper() line in the functions.

Now, let’s write a functions that will check if the sequence is valid. That way we will focus all the logic related to checking for invalid sequences in one part of the code, and we can call it anywhere we need afterwards. So, the seq_features.py looks like this:

def is_valid_sequence(seq):
    """Ensure valid sequence of amino acids."""
    for aa in seq:
        if aa not in "ARNDCQEGHILKMFPSTWYV":
            raise RuntimeError(aa + ' is not a valid amino acid.')


def number_negatives(seq):
    """Number of negative residues a protein sequence"""
    # Convert sequence to upper case
    seq = seq.upper()

    # Check for a valid sequence
    is_valid_sequence(seq)

    # Count E's and D's, since these are the negative residues
    return seq.count('E') + seq.count('D')


def number_positives(seq):
    """Number of positive residues a protein sequence"""
    # Convert sequence to upper case
    seq = seq.upper()

    # Check for a valid sequence
    is_valid_sequence(seq)

    return seq.count('R') + seq.count('K') + seq.count('H')

Now let’s include a two new tests to test_seq_features.py.

def test_number_negatives_for_invalid_amino_acid_anywhere():
    with pytest.raises(RuntimeError) as excinfo:
        seq_features.number_negatives('AZK')
    excinfo.match("Z is not a valid amino acid")


def test_number_positives_for_invalid_amino_acid_anywhere():
    with pytest.raises(RuntimeError) as excinfo:
        seq_features.number_positives('AZK')
    excinfo.match("Z is not a valid amino acid")

!pytest -v -W ignore::DeprecationWarning

============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-5.1.2, py-1.8.0, pluggy-0.13.0 -- /Users/Justin/anaconda3/bin/python
cachedir: .pytest_cache
rootdir: /Users/Justin/Dropbox/git/bebi103_course
collected 12 items

test_seq_features.py::test_number_negatives_single_E_or_D PASSED         [  8%]
test_seq_features.py::test_number_negatives_for_empty PASSED             [ 16%]
test_seq_features.py::test_number_negatives_for_short_sequences PASSED   [ 25%]
test_seq_features.py::test_number_negatives_for_lowercase PASSED         [ 33%]
test_seq_features.py::test_number_negatives_for_invalid_amino_acid PASSED [ 41%]
test_seq_features.py::test_number_positives_single_R_K_or_H PASSED       [ 50%]
test_seq_features.py::test_number_positives_for_empty PASSED             [ 58%]
test_seq_features.py::test_number_positives_for_short_sequences PASSED   [ 66%]
test_seq_features.py::test_number_positives_for_lowercase PASSED         [ 75%]
test_seq_features.py::test_number_positives_for_invalid_amino_acid PASSED [ 83%]
test_seq_features.py::test_number_negatives_for_invalid_amino_acid_anywhere PASSED [ 91%]
test_seq_features.py::test_number_positives_for_invalid_amino_acid_anywhere PASSED [100%]

============================== 12 passed in 0.07s ==============================

There we have it. We are passing all the tests even though we changed our code to accommodate new demands. We can guarantee that it is still working the way it was first intended in addition to the new functionalities.

As an added bonus, we don’t need to write tests related to valid sequence for number_negatives() and number_positives() because these functions are not supposed to be responsible for this task anymore.

That said, refactoring tests is frowned upon and taken VERY seriously by developers; it is a very big responsibility and should be done carefully if ever. Keep on adding tests related to is_valid_sequence(), but do not remove the previous tests already in the suite unless you have thought long and hard about it (and discussed at length with any collaborators on the code base). Refactoring tests usually means you’re making an API change, which you should also think very carefully about.

So, let’s add the exception tests for is_valid_sequence() in test_seq_features.py:

def test_is_valid_sequence_for_invalid_amino_acid():
    with pytest.raises(RuntimeError) as excinfo:
        seq_features.is_valid_sequence('Z')
    excinfo.match("Z is not a valid amino acid")


def test_is_valid_sequence_for_invalid_amino_acid_anywhere():
    with pytest.raises(RuntimeError) as excinfo:
        seq_features.is_valid_sequence('AZK')
    excinfo.match("Z is not a valid amino acid")

and run the tests again.

!pytest -v -W ignore::DeprecationWarning

============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-5.1.2, py-1.8.0, pluggy-0.13.0 -- /Users/Justin/anaconda3/bin/python
cachedir: .pytest_cache
rootdir: /Users/Justin/Dropbox/git/bebi103_course
collected 14 items

test_seq_features.py::test_number_negatives_single_E_or_D PASSED         [  7%]
test_seq_features.py::test_number_negatives_for_empty PASSED             [ 14%]
test_seq_features.py::test_number_negatives_for_short_sequences PASSED   [ 21%]
test_seq_features.py::test_number_negatives_for_lowercase PASSED         [ 28%]
test_seq_features.py::test_number_negatives_for_invalid_amino_acid PASSED [ 35%]
test_seq_features.py::test_number_positives_single_R_K_or_H PASSED       [ 42%]
test_seq_features.py::test_number_positives_for_empty PASSED             [ 50%]
test_seq_features.py::test_number_positives_for_short_sequences PASSED   [ 57%]
test_seq_features.py::test_number_positives_for_lowercase PASSED         [ 64%]
test_seq_features.py::test_number_positives_for_invalid_amino_acid PASSED [ 71%]
test_seq_features.py::test_number_negatives_for_invalid_amino_acid_anywhere PASSED [ 78%]
test_seq_features.py::test_number_positives_for_invalid_amino_acid_anywhere PASSED [ 85%]
test_seq_features.py::test_is_valid_sequence_for_invalid_amino_acid PASSED [ 92%]
test_seq_features.py::test_is_valid_sequence_for_invalid_amino_acid_anywhere PASSED [100%]

============================== 14 passed in 0.13s ==============================

We should write more careful tests for is_valid_sequence() to cover more possible errors than just having a Z in a sequence. This is nice; now we just need to code a single test function for it, in contrast to writing two of them: one for number_negatives() and another for number_positives(). We can add this test:

def test_is_valid_sequence_for_other_invalid_amino_acid_anywhere():
    assert seq_features.is_valid_sequence('ALKSAYGS') is None

    with pytest.raises(RuntimeError) as excinfo:
        seq_features.is_valid_sequence('AZLL')
    excinfo.match("Z is not a valid amino acid")

    with pytest.raises(RuntimeError) as excinfo:
        seq_features.is_valid_sequence('ALLBJ')
    excinfo.match("B is not a valid amino acid")

    with pytest.raises(RuntimeError) as excinfo:
        seq_features.is_valid_sequence('AL%J')
    excinfo.match("% is not a valid amino acid")

And let’s run the tests again.

!pytest -v -W ignore::DeprecationWarning

============================= test session starts ==============================
platform darwin -- Python 3.7.4, pytest-5.1.2, py-1.8.0, pluggy-0.13.0 -- /Users/Justin/anaconda3/bin/python
cachedir: .pytest_cache
rootdir: /Users/Justin/Dropbox/git/bebi103_course
collected 15 items

test_seq_features.py::test_number_negatives_single_E_or_D PASSED         [  6%]
test_seq_features.py::test_number_negatives_for_empty PASSED             [ 13%]
test_seq_features.py::test_number_negatives_for_short_sequences PASSED   [ 20%]
test_seq_features.py::test_number_negatives_for_lowercase PASSED         [ 26%]
test_seq_features.py::test_number_negatives_for_invalid_amino_acid PASSED [ 33%]
test_seq_features.py::test_number_positives_single_R_K_or_H PASSED       [ 40%]
test_seq_features.py::test_number_positives_for_empty PASSED             [ 46%]
test_seq_features.py::test_number_positives_for_short_sequences PASSED   [ 53%]
test_seq_features.py::test_number_positives_for_lowercase PASSED         [ 60%]
test_seq_features.py::test_number_positives_for_invalid_amino_acid PASSED [ 66%]
test_seq_features.py::test_number_negatives_for_invalid_amino_acid_anywhere PASSED [ 73%]
test_seq_features.py::test_number_positives_for_invalid_amino_acid_anywhere PASSED [ 80%]
test_seq_features.py::test_is_valid_sequence_for_invalid_amino_acid PASSED [ 86%]
test_seq_features.py::test_is_valid_sequence_for_invalid_amino_acid_anywhere PASSED [ 93%]
test_seq_features.py::test_is_valid_sequence_for_other_invalid_amino_acid_anywhere PASSED [100%]

============================== 15 passed in 0.07s ==============================

Where do we go from here?

There are tons of details about pytest that will address most issues you will encounter while working on your programs. It is very well documented, so you can use that to develop tests for your code.

Computing environment

%load_ext watermark
%watermark -v -p pytest,jupyterlab

CPython 3.7.4
IPython 7.1.1

pytest 5.1.2
jupyterlab 1.1.4

Copyright note: In addition to the copyright shown below, Davi Ortega contributed to this lecture.