CS 422/522 MAXIMS
I. Problem-Solving Environments
- There are three great branches of science: theory, experiment,
- A computational study is unlikely to lead to real scientific
progress unless the software environment is convenient enough to
encourage one to vary parameters, modify the problem, play around.
- Numerical analysis is the study of the mathematics of the previous
- If the answer is highly sensitive to perturbations,
you have probably asked the wrong question.
- Scientific progress occurs by revolutions and paradigm shifts
on all scales. If Kuhn had written The Structure of Scientific
Revolutions after the popularization of fractals, he would never
have dared to claim there were just two.
- One of the most important of disciplines
is the one that never assumed its logical name:
II. Symbolic Computing
- The big gulf in the mathematical sciences is between the
continuous problems (and people) and the discrete ones.
Most scientists and engineers are in the continuous group,
and most non-numerical computer scientists are in the discrete one.
- No physical constants are known to more than around eleven
digits, and no truly scientific problem requires computation with
much more precision than this.
- Most problems of continuous mathematics
cannot be solved by finite algorithms.
Corollary: if rounding errors vanished, 95% of numerical
analysis would remain.
- Chess is a finite game, hence trivial,
but this fact does not seem to dismay those who play it.
- Just because there's an exact formula doesn't mean
it's necessarily a good idea to use it.
- Symbolic computing is mainly useful when you want a symbolic answer.
III. Software Libraries
- As technology advances, the ingenious ideas that make progress possible
vanish into the inner workings of our machines, where only experts may be
aware of their existence.
Numerical algorithms, being exceptionally uninteresting and
incomprehensible to the public, vanish exceptionally fast.
- Is there an O(n2+epsilon)) algorithm for solving an
n x n system Ax=b ?
This is the biggest unsolved problem in numerical analysis, but
nobody is working on it.
The purpose of computing is insight, not pictures. [after Hamming]
- Twenty years ago, we did not interact with computers graphically;
now, everything is graphical. In the next twenty years an equally
great change will occur as we begin to communicate with machines
- Clouds are not spheres, mountains are not cones,
coastlines are not circles, and bark is not smooth,
nor does lightning travel in a straight line. [Mandelbrot]
- 109 is a kind of magic number, above which new effects emerge. Think
of neurons in the brain---nucleotides in the genetic code---people on
earth---or the number of calculations routinely carried out by a computer
each time you press a button.
- Eventually mankind will solve the problem of consciousness
by deciding that we are
not conscious after all, nor ever were.
V. Parallel Computing
In the long run, our large-scale computations
must inevitably be carried out in parallel.
Nobody really knows how to program parallel computers.
Nobody really knows how the brain works.
In the next century, revolutionary developments will occur in
Advanced technological civilizations can last a few hundreds or maybe
thousands of years, but not millions.
(If this is not so, how can you explain the extraordinary coincidence
that you live right at the dawn of this one? And how can you explain
the fact that despite millions of years for the journey, civilizations
from other parts of the universe have not taken over the Earth?)
Thanks to digital logic and careful error correction, computers
have traditionally behaved deterministically: if you run the program
twice, you get the same answer both times. However, as computing becomes ever
more intelligent and more distributed in the upcoming century,
determinism in any practical sense will fade away. No fully intelligent
system can be expected to give you the same answer twice in a row.
If the state space is huge, the only reasonable way to explore it
is at random.
VI. Program Transformations
Computational mathematics is mainly based on two big ideas:
Taylor series and linear algebra.
In principle, the Taylor series of a function of n variables
involves an n-vector, an n-by-n matrix, an n-by-n-by-n
tensor, and so on. Actual use of orders higher than two, however, is
so rare that the manipulation of matrices is a hundred times better supported
in our brains and in our software tools
than that of tensors.
When the arithmetic is easy and the challenge lies in efficient ordering
of a sequence of operations, computational science
turns to graph theory. Two big examples are automatic differentiation
and direct methods for sparse linear systems of equations.
VII. Web-based Computing
The two biggest world events in the second half of the
20th century were the end of the Cold War in 1989 and the explosion of
the World Wide Web beginning in 1994.
Life on Earth consists of 107 species, each consisting of
1010 individuals, each consisting of 1013 cells,
each consisting of 1014 atoms. (These numbers are very rough!)
Computer codes are better written than genetic ones, since there's a
programmer in the loop, but as they get bigger, this distinction is
Living things are three-dimensional objects, yet they
are constructed by folding up one-dimensional pieces (proteins).
This astonishing method of construction is what makes
repair, reproduction, and natural selection practicable.
(If an infinite intelligence designed an organism from scratch,
by contrast, presumably it would use 3D building blocks.)
Mankind discovered 50 years ago what natural selection discovered
3,800,000,000 years ago.
If you want to manipulate and copy information without any errors,
your logic has to be digital.
Animals are composed of billions of autonomous cells, each
performing specialized functions
and communicating with the others
by prescribed signals. It is almost unimaginable
that large creatures could
have been engineered without such a modular design.
The same conclusion applies in
the engineering of large software systems.
Once we know the human genome, will that knowledge prove useful?
You might as well ask an expert user of a software system,
would it be helpful to have access to the source code?