CS/CiS/Psych/Engri/Music/Film/Dance 1610
Computing in the Arts

Fall, 2008

Instructor: Professor Graeme Bailey


Tentative Syllabus:

The primary technical topics will introduce and explore the applications of stochastic processes, geometries and symmetries, group actions, iterative systems, sampling, and synthesis to the four example art forms of music, poetry, two-dimensional art and interactive ‘installations’. Additionally some attention will be given, albeit abstractly, to sensor and feedback systems and to some aspects of Human Computer Interfaces relating to the creative process. Since this is a 100-level course intended partly as a tempting survey of the field, some of this will be handled in a rather matter-of-fact manner (no discussion of convergence criteria for Fourier Series!!, nor even any mathematically explicit use of calculus). Because of the helpful relative discreteness of musical notation, most initial examples will be from music and poetry, only moving to the additional richness of graphic media after the underlying ideas have been grasped fully. Somewhat related courses at peer institutions (Berkeley, Stanford, MIT, Washington, York (UK), Paris) have descriptives such as “algorithmic [art] is [art] produced through processes which are pre-determined”, “the [artist] deals more often with macro-structural principles rather than fine detail”, algorithmic techniques often form the basis of interactive works”. Success in this course will be assessed primarily by homeworks and a final project, although there will be a mid-term exam covering the fundamental concepts about 2/3 of the way through the semester in order to ensure that any follow-up courses can assume a certain core knowledge. The course has no calculus or programming pre-requisites.

The primary areas which may be addressed in the course are:

Composition

- probability - controlled randomness (Markov processes mostly)

- geometry - controlled symmetry (tessellations (visual and musical), rotations/reflections, projective plane

- iteration - sending and feedback

- variations and metamorphoses - a homotopy of artistic elements

What is form?

- rules, structure, and rule breaking

- understanding the context (cultural and other influences)

Perception

- what and how do we perceive our environment?

- understanding how we function!

- the role of experiments

Synthesis

- simulating extant instruments/media

- modifying extant instruments/media

- creating new soundscapes/visual experiences/environments

- playing with the audience's/viewer's mind

- formal techniques

Sampling

- sounds/media (both analytical and statistical - though at a very elementary and intuitive level)

- sampling full compositions/complete artworks

Interfaces

- typing into the computer!

- pianolike keyboard/simulating string and wind inputs/tablets/simulating brush and palette knife inputs

- the importance of tactile feedback

- creating new interfaces - the CAVE


A putative syllabus (still subject to significant change - in particular I still need to rework the wk 8+ material from sampling/synthesis onwards to decide what should stay and what should go to avoid overwhelming the students yet give them a good taster for the area. Also this year there will be a terrific opportunity to substantially enrich the course with real knowledge about perception and cognition related to this whole area):

Wk 1

what is probability? gentle intro to stochastic processes/Markov chains. Discuss the ideas for building a stochastic engine to generate poetry (free-form and rhyming) and simple tunes - use polygonal tops!

First steps in considering perception/cognition aspects, possibly relating to approaches to 'tuning' the results of the first experiment to understand how to go about improving the 'output'.

Wk 2

basics of music notation, build a stochastic composition engine:

1) notes only (melody), 2) 'tune probabilities', 3) what makes an effective tune (cf part 6)?

Possibly discuss very easy programming to build applications: #1: though probably defer this until later.

Start some very easy Java programming

Wk 3

More easy JAva -- how to draw graphics and how to make sound (java MIDI)

stochastic composition engine: 4) composing using deformable rules (write a not-very-good fugue!), 5) keys, scales, chords and basics of harmony,

6) ‘re-tune’ probabilities thinking of harmonic leading

Wk 4

apply this engine to the visual arts: 1) control lines, regions, colour, 2) ‘tune’ probabilities for effect, 3) insights into human creativity,

Very easy programming to build applications: #2

Overview and pointers to resources addressing human visual and auditory perception

Homework – develop visualizations for your earlier music

Wk 5

further mechanisms and applications to clumps of artistic elements instead of elementary objects:

1) geometry and symmetry in the plane, rotations and reflections - examples in music and art

a ) music: Lou Harrison’s rhythmicals, melodicals and interval controls, also inversion, retrograde, etc

b) art: manipulating patterns of colour and texture, Harold Cohen’s “Aaron” – the robotic artist

2) tessellations from a fundamental region - examples in music and art

Wk 6

further mechanisms: 3) deforming shapes (controlled distortion) – examples of metamophoses in music and art

4) easy projective geometry in the plane – examples in art

Wk 7

iterative processes with examples: 1) basic mathematical principles, 2) gentle targeted intro to dynamical systems, 3) building interactivity into your art (text, sound and visual)

Wk 8

Sampling/synthesis: 1) sound: a) pragmatic approach to Fourier, b) granular synthesis, etc, c) overview of statistical sampling with application,

Wk 9

easy programming to build applications: #3: intro to CSound as explorations in musical synthesis (or else use MAX/MSP as an easier environment) exploring new soundscapes (beyond the ‘BBC radiophonic workshop’)

Wk 10

Sampling/Synthesis 2) poetry: fake Shakespeare, 3) art: sample and recompose Mondrian/Breugel, 4) discuss sampling and faking Bach/Mozart/other – illustrate with examples and discuss history

Wk 11

music specific stuff: 1) more on scales and temperament, 2) tuning of ‘non-physics-ideal’ instruments (inharmonicity), 3) influences of instrument choices on musical scales and style, 4) new sounds (and cultural preconceptions) – extended discussion of ‘what is effective?’

Wk 12

art specific stuff: 1) colour and colour perception by juxtaposition, 2) colour behaviour via different media, 3) interactive installations, 4) employing sound as part of the visual art experience, 5) new art specific to the new digital medium (screens, installations, the ‘CAVE’, immersive environments)

Wk 13

expanding the creative process: the role of Human Machine Interfaces as constraints or facilitators (exploring both human/computer interfaces and interfaces to musical instruments, writing media, drawing/painting media, and sculpting)

Wk 14

project focus