Package redempt.redlex.data

Class TokenType

java.lang.Object

redempt.redlex.data.TokenType

Direct Known Subclasses:

CharGroupToken, CharSetToken, ChoiceToken, EndOfFileToken, ListToken, NotToken, PlaceholderToken, RepeatingToken, StringChoiceToken, StringToken

public abstract class TokenType
extends Object

Represents a type of token which can be used to create a Lexer

Constructor Summary

Constructors

Constructor	Description
TokenType(String name)	Create a new TokenType with the given name

Method Summary

Modifier and Type	Method	Description
protected abstract List<Character>	calcFirstCharacters()
abstract boolean	characterMatches(String input, int pos, int offset)
Token	findForward(String str, int pos, LexContext ctx)
List<Character>	getFirstCharacters()
Lexer	getLexer()
String	getMessage()
String	getName()
abstract boolean	lengthMatches(int length)
abstract int	maxLength()
abstract int	minLength()
boolean	nameMatches(String name)	Check whether the name of this TokenType matches the given name
void	replacePlaceholders(Map<String,TokenType> tokens)
void	setLexer(Lexer lexer)	Sets the Lexer this TokenType and its children belong to
void	setName(String name)	Sets the name of this TokenType
String	toString()
void	walk(Consumer<TokenType> each)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

TokenType

public TokenType(String name)

Create a new TokenType with the given name

Parameters:

name - The name for this TokenType

Method Details

setLexer

public void setLexer(Lexer lexer)

Sets the Lexer this TokenType and its children belong to

Parameters:

lexer - The Lexer

getLexer

public Lexer getLexer()

Returns:

The Lexer this TokenType belongs to

nameMatches

public boolean nameMatches(String name)

Check whether the name of this TokenType matches the given name

Parameters:

name - The name to compare

Returns:

Whether this TokenType's name matches the given name

getName

public String getName()

Returns:

The name of this TokenType

setName

public void setName(String name)

Sets the name of this TokenType

Parameters:

name - The name to set

findForward

public Token findForward(String str,
 int pos,
 LexContext ctx)

getMessage

public String getMessage()

replacePlaceholders

public void replacePlaceholders(Map<String,TokenType> tokens)

walk

public void walk(Consumer<TokenType> each)

characterMatches

public abstract boolean characterMatches(String input,
 int pos,
 int offset)

lengthMatches

public abstract boolean lengthMatches(int length)

minLength

public abstract int minLength()

maxLength

public abstract int maxLength()

calcFirstCharacters

protected abstract List<Character> calcFirstCharacters()

getFirstCharacters

public List<Character> getFirstCharacters()

Returns:

A list of all the characters which may appear as the first character of this token. Includes null if this token may be zero-length.

toString

public String toString()

Overrides:

toString in class Object