Skip to main content
more options

Testing

Testing is the process of running a program against "test cases" in order to get some evidence of the correctness of the program. If an error is discovered, then the process of debugging attempts to locate the error and fix it.

This topic is not sexy. In fact, many would call it dull. Nevertheless, you can save yourself a lot of time and effort if you follow some simple testing principles and strategies. It is worth spending some initial time learning about the principles and strategies and practicing them on your programming assignments.

We suggest that you read Gries/Gries pp. 385–389, about testing and creating test cases, and Gries/Gries pp. 392–394 about approaches to creating test cases. Also, read carefully the ProgramLive CD, pp. 14-1 and 14-3, and watch the lectures on those pages.

Below, we summarize a bit of the material; the text goes into more detail.

Two guidelines for testing (Gries/Gries, p. 386)

  • Guideline. Test each method thoroughly as it is completed! Develop the methods in a class in an order that allows each method, or small group of methods, to be tested completely before proceeding to the next one(s). Do this for two reasons:
    • You gain confidence in what you are doing early on. You have some parts of the program that are written and tested.
    • You can rely on the correctness of previously completed methods (if you wrote them and tested them properly). This makes debugging easier if you uncover an error.
  • Guideline. When both writing and testing a method, understand what the method is supposed to do. This means that the method specification should be written before the method body.

Five maxims for creating test cases (Gries/Gries pp. 387–388)

  1. Test early and often. Again, the sooner you test, the more confidence you gain and the easier later debugging becomes.
  2. Test only one thing at a time. Try to do too much at one time, and you may lose track of what you are testing and have difficulty covering all cases.
  3. Test 0, 1, many. Apply this to both input and output. For example, if there is a loop, have test cases so that it (1) executes 0 iterations, (2) 1 iteration, and (3) many iterations. If there is an output list, have test cases that end up with (1) an empty output list, (2) an output list of size 1, and (3) an output list with many elements.
  4. Test null, beginning, middle, and end.
  5. Verify the documentation. As you test, ask yourself questions that you might not normally think of when writing the code, and make sure the documentation answers these questions.

Approaches to creating test cases (Gries/Gries pp. 392–393)

  1. Exhaustive testing: Test all possible inputs to a program. In general, this is infeasible, and we don't discuss it further.
  2. Blackbox testing or functional testing: Look only at the specification of a program unit (e.g. a method) to be tested, and not at the code of the program unit.
  3. Whitebox testing or structural testing: Use the structure of a program in developing test cases. This method is used most often, especially in programming courses and in programs developed by small teams. And it is the kind of testing you should do in this course. An important point here is to provide test coverage: each statement or expression in a program should be exercised by at least one test case. For example, any if-statement requires at least two test cases, one for the if-condition being false and the other for it being true.