This document describes the major changes made to the Polyglot framework
for version 2.0 and how to upgrade extensions written for Polyglot 1.3.

The pass scheduling code was rewritten for Polyglot 2.0.  The goal was
to simplify the disambiguation passes, reduce the need for barrier
passes, and clean up the invariants assumed by the passes.

As an example, the Jx extension to Polyglot 1.3 had the following
disambiguation passes:

        create packages
        barrier
        disambiguate packages
        barrier
        type builder
        barrier
        disambiguate supertypes
        add dummy fields
        barrier
        disambiguate member signatures
        remove dummy fields
        barrier
        disambiguate method bodies
        add members
        type check
        ...

The new schedule has the following passes:

        type builder
        barrier
        disambiguate
        type check
        ...

Not only is the schedule much shorter, but the compiler should use less
memory as well because it uses fewer barriers (I'm still doing performance
tuning, though).

The changes to the scheduler are not backward compatible and some
porting effort is required.  Porting small extensions (e.g., PAO) takes
a few minutes.  Porting Coffer took a few hours.  Larger extensions
(e.g., Jif, Jx) may take a few days or a week.

The Scheduler
-------------

The new scheduler works by attempting to satisfy _goals_.
Examples of goals are:

Parsed(Job)
        - Satisfied if the job's source file has been parsed
MembersAdded(ClassType)
        - Satisfied if members have been added to the class's type object
SupertypesResolved(ClassType)
        - Satisfied if the class's supertypes have been disambiguated
SignaturesResolved(ClassType)
        - Satisfied if member signatures of the class have been disambiguated
VisitorGoal(Job, Visitor) -
        - Satisfied if visitor has been run over job's AST
CodeGenerated(Job)
        - Satisfied if code has been generated from the job's AST

A goal may be dependent on other goals.

Goals can be satisfied by running passes.  Most passes just run a
visitor over the AST.  If no errors are encountered when running a pass,
the pass is said to succeed.  Even if a pass succeeds, the goal may not
be satisfied because running the pass may have caused the goal to become
dependent upon another goal, which is not yet satisfied.  For example,
the disambiguation pass may resolve a class name to a particular class
C, but may not be able to lookup a member of that class because the C's
members have not yet been added to the type object for C.  In this case,
the disambiguation pass is rescheduled to run after the members have
been added.

There are two types of dependencies between goals: prerequisite and
corequisite dependencies.  If a goal has a prerequisite dependency, that
dependency must be satisfied before the goal is attempted.  The compiler
will throw an InternalCompilerError if there is a cycle of prerequisite
dependencies.  If a goal has a corequisite dependency, running a pass for
the dependency may help the dependent goal be satisfied.  Having a cycle
of corequisite dependencies may or may not cause the compiler to go into
an infinite loop.  This situation is detected at run time and the
compiler aborts.  Infinite loop detection is still buggy.

If, while a pass is running, a dependency cannot be satisfied, the
pass should throw a SchedulerException (or more usually a
MissingDependencyException The scheduler catches the exception and adds
the goal back onto the worklist and adds the missing dependency, if any,
to the dependency graph.  Optionally, a pass may catch the exception,
but it must mark the goal as unreachable on this run (call
Goal.setUnreachableThisRun) so that it will be added back to the
worklist; the pass must also add any missing dependencies.


Specific changes
----------------

Declarations (e.g., ClassDecl, MethodDecl, LocalDecl, Formal, etc)
should create corresponding TypeObjects exactly once, in the buildTypes
pass.  Class members should also be added during buildTypes.  The new
TypeObjects may be ambiguous (i.e., they may point to ambiguous
TypeObjects).  Later passes should imperatively update the TypeObjects
to be unambiguous.  Other node types (e.g., TypeNode, Call, Local, etc)
have references to the same TypeObjects created by the declarations.

Since TypeObjects used in declarations should be updated imperatively,
setter methods are needed for new TypeObjects added by an extension.
Only TypeObjects that are directly shared by more than one AST node
(e.g., Type, MethodInstance, FieldInstance, LocalInstance, etc) require
the setter methods.

Declaration nodes should also implement isDisambiguated().  This method
should return true if the TypeObject for the node has been completely
disambiguated.  It should not recurse on the node's children (a visitor
does that).

Pass.ID is no longer used.  It has been subsumed by Goal.

ExtensionInfo.passes() has been removed.  Extensions should override
the scheduler to modify the dependency graph for new and inherited
goals.  See the PAO and Coffer extensions for examples.

HaltingVisitor is deprecated; return a non-null Node from
NodeVisitor.override() to prune a visitor traversal.  InnerJob has been
removed and SourceJob has been merged into Job.  Job spawning has been
removed; just run visitors explicitly over the AST subtree.  Be sure to
set the new visitor's Context if needed.

If new class member types are added, extend LazyClassInitializer with
init methods.  Extend SchedulerClassInitializer to enforce scheduler
dependencies and DeserializedLazyClassInitializer also.  Override
TypeSystem_c.defaultClassInitializer() and
TypeSystem_c.deserializedClassInitializer().  Also override
ParsedClassType_c.signaturesResolved().

Lexer_c now takes a Source object, not the name of a source.
Adjust *.flex and ExtensionInfo accordingly.  It's probably easiest
just to copy over java.flex and reapply your changes.

    old ExtensionInfo:

      Lexer lexer = new Lexer_c(reader, source.name(), eq);

    new ExtensionInfo:

      Lexer lexer = new Lexer_c(reader, source, eq);

