Errors and Exception handling

---

# Colab setup ------------------
import os, sys, subprocess
if "google.colab" in sys.modules:
    cmd = "pip install --upgrade watermark"
    process = subprocess.Popen(cmd.split(), stdout=subprocess.PIPE)
# ------------------------------

import warnings

---

So far, we have encountered errors when we did something wrong. For example, when we tried to change a character in a string, we got a TypeError.

my_str = 'AGCTATC'
my_str[3] = 'G'

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-d08010a7add0> in <module>
      1 my_str = 'AGCTATC'
----> 2 my_str[3] = 'G'

TypeError: 'str' object does not support item assignment

In this case, the TypeError indicates that we tried to do something that is legal in Python for some types, but we tried to do it to a type for which it is illegal (strings are immutable). In Python, an error detected during execution is called an exception. We say that the interpreter “raised an exception.” There are many kinds of built-in exceptions, and you can find a list of them, with descriptions here. You can write your own kinds of exceptions, but we will not cover that.

In this lesson, we will investigate how to handle errors in your code. Importantly, we will also touch on the different kinds of errors and how to avoid them. Or, more specifically, you will learn how to use exceptions to help you write better, more bug-free code.

Kinds of errors

In computer programs, we can break down errors into three types.

Syntax errors

A syntax error means you wrote something nonsensical, something the Python interpreter cannot understand. An example of a syntax error in English would be the following.

Sir Tristram, violer d’amores, fr’over the short sea, had passen-core rearrived from North Armorica on this side the scraggy isthmus of Europe Minor to wielderfight his penisolate war: nor had topsawyer’s rocks by the stream Oconee exaggerated themselse to Laurens County’s gorgios while they went doublin their mumper all the time: nor avoice from afire bellowsed mishe mishe to tauftauf thuartpeatrick: not yet, though venissoon after, had a kidscad buttended a bland old isaac: not yet, though all’s fair in vanessy, were sosie sesthers wroth with twone nathandjoe.

This is recognizable as English. In fact, it is the second sentence of a very famous novel (Finnegans Wake by James Joyce). Clearly, many spelling and punctuation rules of English are violated here. To many of us, it is nonsensical, but I do know of some people who have read the book and understand it. So, English is fairly tolerant of syntax errors. A simpler example would be

Data anlysis is fun!

This has a syntax error (“anlysis” is not in the English language), but we understand what it means. A syntax error in Python would be this:

my_list = [1, 2, 3

We know what this means. We are trying to create a list with three items, 1, 2, and 3. However, we forgot the closing bracket. Unlike speakers of the English language, the Python interpreter is not forgiving; it will raise a SyntaxError exception.

my_list = [1, 2, 3

  File "<ipython-input-3-34ae6d6cd500>", line 1
    my_list = [1, 2, 3
                      ^
SyntaxError: unexpected EOF while parsing

Syntax errors are often the easiest to deal with, since the program will not run at all if any are present.

Runtime errors

Runtime errors occur when a program is syntactically correct, so it can run, but the interpreter encountered something wrong. The example at the start of the tutorial, trying to change a character in a string, is an example of a runtime error. This particular one was a TypeError, which is a more specific type of runtime error. Python does have a RuntimeError, which just indicates a generic runtime (non-syntax) error.

Runtime errors are more difficult to spot than syntax errors because it is possible that a program could run all the way through without encountering the error for some inputs, but for other inputs, you get an error. Let’s consider the example of a simple function meant to add two numbers.

def add_two_things(a, b):
    """Add two numbers."""
    return a + b

Syntactically, this function is just fine. We can use it and it works.

add_two_things(6, 7)

13

We can even add strings, even though it was meant to add two numbers.

add_two_things('Hello, ', 'world.')

'Hello, world.'

However, when we try to add a string and a number, we get a TypeError, the kind of runtime error we saw before.

add_two_things('a string', 5.7)

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-26b1fde01f0b> in <module>
----> 1 add_two_things('a string', 5.7)

<ipython-input-4-833715e51f11> in add_two_things(a, b)
      1 def add_two_things(a, b):
      2     """Add two numbers."""
----> 3     return a + b

TypeError: can only concatenate str (not "float") to str

Semantic errors

Semantic errors are perhaps the most nefarious. They occur when your program is syntactically correct, executes without runtime errors, and then produces the wrong result. These errors are the hardest to find and can do the most damage. After all, when your program does not do what you designed it to do, you want it to scream out with an exception!

Following is a common example of a semantic error in which we change a mutable object within a function and then try to reuse it.

# A function to append a list onto itself, with the intention of
# returning a new list, but leaving the input unaltered
def double_list(in_list):
    """Append a list to itself."""
    in_list += in_list
    return in_list

# Make a list
my_list = [3, 2, 1]

# Double it
my_list_double = double_list(my_list)

# Later on in our program, we want a sorted my_list
my_list.sort()

# Let's look at my_list:
print('We expect [1, 2, 3]')
print('We get   ', my_list)

We expect [1, 2, 3]
We get    [1, 1, 2, 2, 3, 3]

Yikes! We changed my_list within the function unintentionally. Question: How would you re-rewrite ``double_list()`` to avoid this issue?

Handling errors in your code

If you have a syntax error, your code will not even run. So, we will assume we are without syntax errors in this discussion on how to handle errors. So, how can we handle runtime errors? In most use cases, we just write our code and let the Python interpreter tell us about these exceptions. However, sometimes we want to use the fact that we know we might encounter a runtime error within our code. A common example of this is when importing modules that are convenient, but not essential, for your code to run. Errors are handled in your code using a try statement.

Let’s try importing a module that computes GC content. This doesn’t exist, so we will get an ImportError.

import gc_content

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
<ipython-input-9-b2d775210337> in <module>
----> 1 import gc_content

ModuleNotFoundError: No module named 'gc_content'

Now, if we had the gc_content module, we would like to use it. But if not, we will just hand-code a calculation of the GC content of a sequence. We use a try statement.

# Try to get the gc_content module
try:
    import gc_content
    have_gc = True
except ImportError as e:
    have_gc = False
finally:
    # Do whatever is necessary here, like close files
    pass

seq = 'ACGATCTACGATCAGCTGCGCGCATCG'

if have_gc:
    print(gc_content(seq))
else:
    print(seq.count('G') + seq.count('C'))

16

The program now runs just fine! The try statement consists of an initial try clause. Everything under the try clause is attempted to be executed. If it succeeds, the rest of the try statement is skipped, and the interpreter goes to the seq = ... line.

If, however, there is an ImportError, the code within the except ImportError as e clause is executed. The exception does not halt the program. If there is some other kind of error other than an ImportError, the interpreter will raise an exception after it does whatever code is in the finally clause. The finally clause is useful to tidy things up, like closing open file handles. While it is possible for a try statement to handle any generic exception by not specifying ImportError as e, it is good practice to explicitly specify the exception(s) that you anticipate in try statements as shown here. In this case, we only want to have control over ImportErrors. We want the interpreter to scream at us for any other, unanticipated errors.

Issuing warnings

We may want to issue a warning instead of silently continuing. For this, the warnings module from the standard library is useful. We use the warnings.warn() method to issue the warning.

# Try to get the gc_content module
try:
    import gc_content
    have_gc = True
except ImportError as e:
    have_gc = False
    warnings.warn(
        'Failed to load gc_content. Using custom function.',
        UserWarning
    )
finally:
    pass

seq = 'ACGATCTACGATCAGCTGCGCGCATCG'

if have_gc:
    print(gc_content(seq))
else:
    print(seq.count('G') + seq.count('C'))

16

<ipython-input-11-6702ed84bda0>:7: UserWarning: Failed to load gc_content. Using custom function.
  warnings.warn(

Normally, we would use an ImportWarning, but those are ignored by default, so we have used a UserWarning.

Checking input

It is often the case that you want to check the input of a function when it is called to ensure that it will work properly. In other words, you want to anticipate errors that the user (or you) might make in running your function, and you want to give descriptive error messages. For example, let’s say you are writing a code that processes protein sequences that contain only the 20 naturally occurring amino acids represented by their one-letter abbreviation. You may wish to check that the amino acid sequence is legitimate. In particular, the letters B, J, O, U, X, and Z, are not valid abbreviations for standard amino acids. (We will not use the ambiguity code, e.g. B for aspartic acid or asparagine, Z for glutamine or glutamic acid, or X for any amino acid.)

To illustrate the point, we will write a simple function that converts the sequence of one-letter amino acids to the three-letter abbreviation. We’ll use the dictionary that converts single-letter amino acid codes to triple letter that we encountered in our lesson on dictionaries.

aa_dict = {
    "A": "Ala",
    "R": "Arg",
    "N": "Asn",
    "D": "Asp",
    "C": "Cys",
    "Q": "Gln",
    "E": "Glu",
    "G": "Gly",
    "H": "His",
    "I": "Ile",
    "L": "Leu",
    "K": "Lys",
    "M": "Met",
    "F": "Phe",
    "P": "Pro",
    "S": "Ser",
    "T": "Thr",
    "W": "Trp",
    "Y": "Tyr",
    "V": "Val",
}

def one_to_three(seq):
    """
    Converts a protein sequence using one-letter abbreviations
    to one using three-letter abbreviations.
    """
    # Convert seq to upper case
    seq = seq.upper()

    aa_list = []
    for amino_acid in seq:
        # Check if the `amino_acid` is in our dictionary
        if amino_acid not in aa_dict.keys():
            raise RuntimeError(f'{amino_acid} is not a valid amino acid')
        # Add the `amino_acid` to our aa_list
        aa_list.append(aa_dict[amino_acid])

    # Return the amino acids, joined together, with a dash as a separator.
    return '-'.join(aa_list)

So, if we put in a legitimate amino acid sequence, the function works as expected.

one_to_three('waeifnsdfklnsae')

'Trp-Ala-Glu-Ile-Phe-Asn-Ser-Asp-Phe-Lys-Leu-Asn-Ser-Ala-Glu'

But, it we put in an improper amino acid, we will get a descriptive error.

one_to_three('waeifnsdfzklnsae')

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-14-96dc1e0f73c2> in <module>
----> 1 one_to_three('waeifnsdfzklnsae')

<ipython-input-12-20aa82b4f553> in one_to_three(seq)
     34         # Check if the `amino_acid` is in our dictionary
     35         if amino_acid not in aa_dict.keys():
---> 36             raise RuntimeError(f'{amino_acid} is not a valid amino acid')
     37         # Add the `amino_acid` to our aa_list
     38         aa_list.append(aa_dict[amino_acid])

RuntimeError: Z is not a valid amino acid

Good code checks for errors and gives useful error messages. We will use exception handling extensively when we go over test driven development in future lessons.

Assertion errors

As discussed in the lecture about test-driven development, we often want to check to see if a condition is true. The assert statement provides a convenient method for doing this. As usual, this is best seen by example.

my_str = 'hello'

assert len(my_str) == 5

Notice that nothing happed. This is because len(my_str) is indeed 5. If we go it wrong, though, we get an AssertionError.

my_str = 'hello, world.'

assert len(my_str) == 5

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-16-251e188ec005> in <module>
      1 my_str = 'hello, world.'
      2
----> 3 assert len(my_str) == 5

AssertionError:

The assert statement consists of the word assert followed by an expression that evaluates to a Boolean. Optionally, this statement can be followed by a comma and a string that is printed as the assertion error.

my_str = 'hello, world.'

assert len(my_str) == 5, '`my_str` is not of length 5.'

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
<ipython-input-17-e7caac770083> in <module>
      1 my_str = 'hello, world.'
      2
----> 3 assert len(my_str) == 5, '`my_str` is not of length 5.'

AssertionError: `my_str` is not of length 5.

Assertion errors are important errors in Python because we use them extensively in testing. We typically have lots of assert statements of expressions that should be truth, and if we get an AssertionError, we know the test failed.

Computing environment

%load_ext watermark
%watermark -v -p jupyterlab

CPython 3.8.3
IPython 7.16.1

jupyterlab 2.1.5