Skip to main content

Precision

Precision Matters in this Assignment!

We are going to be using a finite number of bits to represent and perform calculations on continuous quantities. This means different operations that would result in the same value if we performed them with infinite precision may end up producing very slightly different results. For this reason, when testing for equality we often need to allow for a little bit of slack (i.e., close enough to equal should be considered equal). In general, this is where the Precision class in AniGraph comes in.

The Precision Class

The Precision class in AniGraph contains a few helper functions for dealing with precision in your calculations. We encourage you to look at the class to see how it works. In general, it simply adds some epsilon value of tolerance to different calculations. The advantage of using our functions for this will be that we have the option of adjusting epsilon when we test your code, which will make it possible to be more lenient in some cases. Some relevant functions are:

  • Precision.epsilon is the tolerance used for calculations
  • Precision.PEQ(a:number, b:number) tests to see if a and b are equal to within some threshold epsilon.
  • Precision.isTiny(a: number) tests to see if a is less than or equal to epsilon.

So, if you want to test to see if some variable a is equal to zero, you should probably use something like

return Precision.PEQ(a, 0);