Package redempt.redlex.data

Class Token

java.lang.Object
redempt.redlex.data.Token
Direct Known Subclasses:
ObjectToken
public class Token extends Object
Represents a Token created by a Lexer. Acts as a node in a tree.

  • Field Details

    • EMPTY

      public static Token[] EMPTY

    • value

      protected String value

  • Constructor Details

  • Method Details

    • setChildren

      public void setChildren(Token[] children)
      Sets the children of this Token
      Parameters:
      children - The children of this Token

    • getChildren

      public Token[] getChildren()
      Returns:
      The children of this Token

    • getNext

      public Token getNext()
      Returns:
      The next sibling of this Token, or null if none exists

    • getPrevious

      public Token getPrevious()
      Returns:
      The previous sibling of this Token, or null if none exists

    • getParent

      public Token getParent()
      Returns:
      The parent of this Token, or null if none exists

    • getIndex

      public int getIndex()
      Returns:
      The index of this Token in the array of its parent's children

    • getType

      public TokenType getType()
      Returns:
      The type of this Token

    • getValue

      public String getValue()
      Returns:
      The String value of this Token

    • getBaseString

      public String getBaseString()
      Returns:
      The base String this token was parsed from

    • length

      public int length()
      Returns:
      The length of this Token's String value

    • getEnd

      public int getEnd()
      Returns:
      The ending index of this Token

    • getStart

      public int getStart()
      Returns:
      The starting index of this Token

    • setValue

      public void setValue(String value)
      Sets the String value of this Token
      Parameters:
      value - The String value to set

    • joinChildren

      public String joinChildren(String sep)
      Joins the String values of the children of this node separated by a delimiter
      Parameters:
      sep - The delimiter to separate values with
      Returns:
      The joined string

    • joinLeaves

      public String joinLeaves(String sep)
      Joins the leaf nodes of this subtree separated by a delimiter
      Parameters:
      sep - The delimiter to separate values with
      Returns:
      The joined string

    • remove

      public void remove()
      Removes this token from its parent

    • removeChildren

      public void removeChildren(Token... children)
      Removes the given children from this token
      Parameters:
      children - The children to remove

    • removeChildren

      public void removeChildren(int start, int end)
      Removes a range of children from this token by index
      Parameters:
      start - The starting index to remove children from, inclusive
      end - The ending index to remove children to, exclusive

    • replaceWith

      public Token replaceWith(Object o)
      Replaces this token in its parent with a token wrapper for the given object, or a token if the passed object is a token
      Parameters:
      o - The object to replace this token with
      Returns:
      The Token this one was replaced with

    • replaceParent

      public void replaceParent()
      Replaces this token's parent with itself

    • liftChildren

      public void liftChildren()
      Removes this token from its parent, replacing it with its children

    • firstByName

      public Token firstByName(String name)
      Find the first token in the tree by name (breadth-first)
      Parameters:
      name - The name of the token to search for
      Returns:
      The first token found, or null

    • allByName

      public List<Token> allByName(TraversalOrder order, String name)
      Find all tokens in the tree by name
      Parameters:
      order - The order to traverse the tree in
      name - The name of the token to search for
      Returns:
      All tokens found

    • allByName

      public List<Token> allByName(String name)
      Find all tokens in the tree by name, breadth-first
      Parameters:
      name - The name of the token to search for
      Returns:
      All tokens found

    • allByNames

      public Map<String,List<Token>> allByNames(String... names)
      Find all tokens in the tree by any number of names and create a map, breadth-first
      Parameters:
      names - The names to look for
      Returns:
      A map of names to the tokens with those names

    • allByNames

      public Map<String,List<Token>> allByNames(TraversalOrder order, String... names)
      Find all tokens in the tree by any number of names and create a map
      Parameters:
      order - The order to traverse the tree in
      names - The names to look for
      Returns:
      A map of names to the tokens with those names

    • walk

      public void walk(TraversalOrder order, Consumer<Token> forEachNode)
      Walks this Token tree in the given order
      Parameters:
      order - The order to walk the tree in
      forEachNode - The lambda to apply to each token

    • getObject

      public Object getObject()
      Returns:
      Nothing, unless this is an ObjectToken

    • splitChildren

      public List<List<Token>> splitChildren(String name)
      Splits child tokens on a given token by name
      Parameters:
      name - The name of the token to split on
      Returns:
      A 2d list of tokens split by the given token

    • cull

      public void cull(TokenFilter... filters)
      Culls tokens in the tokens which are descendants of this one using filters.
      Parameters:
      filters - The filters to cull tokens with

    • addChildren

      public void addChildren(Token... tokens)
      Appends the given tokens to this token's children
      Parameters:
      tokens - The tokens to add

    • toString

      public String toString()
      Overrides:
      toString in class Object