javafe.parser
Class Token

java.lang.Object
  javafe.parser.Token

Direct Known Subclasses:

Lex

public class Token

extends java.lang.Object

The Token class defines a set of fields that describe lexical tokens.

Allocating an individual Token object for every token consumes a lot of memory. Instead, our front end tries to reuse Token instances as much as possible. Thus, for example, Lex is a subclass of Token; Lex returns information about the current token not by allocating a new Token object but rather by filling in the Token fields of itself. To facilitate the reuse of Token instances, Token has a copyInto method.

Field Summary

java.lang.Object	auxVal Auxillary information about the token.
static int	CLEAR
int	endingLoc The location of the last character of the token.
Identifier	identifierVal Identifier represented by the token.
int	startingLoc The location of the first character of the token.
int	ttype Integer code giving the kind of token.

Constructor Summary

Token()
* Creation: * *

Method Summary

void	clear() Clear the current token.
Token	copyInto(Token dst) Copy all the fields of this into dst.
java.lang.String	ztoString() Return a representation of this suitable for debug output.
void	zzz() Check the invariants of this.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ttype

public int ttype

Integer code giving the kind of token.

To clear a token, set this field to Token.CLEAR and set identifierVal and auxVal to null, as is done in the clear() method.

See Also:

TagConstants

CLEAR

public static final int CLEAR

See Also:

Constant Field Values

startingLoc

public int startingLoc

The location of the first character of the token.

endingLoc

public int endingLoc

The location of the last character of the token. (This value isn't "off-by-one" right now.)

identifierVal

public Identifier identifierVal

Identifier represented by the token. Must be non-null if ttype is TagConstants.IDENT.

auxVal

public java.lang.Object auxVal

Auxillary information about the token. In the case of literal tokens, this field holds the value of token. In particular, if ttype is one of the codes on the left of the following table, then auxVal must be an instance of the class on the right:

TagConstants.INTLIT	Integer
TagConstants.LONGLIT	Long
TagConstants.FLOATLIT	Float
TagConstants.DOUBLELIT	Double
TagConstants.STRINGLIT	String
TagConstants.CHARLIT	Integer
TagConstants.LEXICALPRAGMA	LexicalPragma
TagConstants.MODIFIERPRAGMA	ModifierPragma
TagConstants.STMTPRAGMA	StmtPragma
TagConstants.TYPEDECLELEMPRAGMA	TypeDeclElemPragma

For the various pragmas, auxVal may be null, but for the literals it may not be.

Constructor Detail

Token

public Token()

* Creation: * *

Method Detail

clear

public void clear()

Clear the current token.

copyInto

public final Token copyInto(Token dst)

Copy all the fields of this into dst. For convenience, returns dst.

ztoString

public java.lang.String ztoString()

Return a representation of this suitable for debug output.

zzz

public void zzz()

Check the invariants of this.