Automatic Differentiation
(AD) is a technique for augmenting computer programs with statements for
the computation of derivatives based on the chain rule of differential
calculus. Due to the associativity of the chain rule, the problem of
computing the Jacobian dy/dx defined by the inputs x and outputs y of a
computer program with minimal complexity is NP-hard. We give an
introduction into the heuristics developed for AD, motivate hierarchical
approaches that allow to take advantage of program or algorithmic
structure, and highlight current efforts for the development of AD
tools. Lastly, we show the potential for parallelism created by AD and
comment on the impact on scientific computing due to multicore chips
which are providing substantial thread-based parallelism in a blade form
factor.