Package redempt.redlex.processing

Interface TokenFilter

public interface TokenFilter
Filters Tokens to remove them from a tree

  • Method Details

    • removeEmpty

      static TokenFilter removeEmpty()
      Returns:
      A token filter that removes all empty tokens

    • flatten

      static TokenFilter flatten()
      Returns:
      A token filter that removes all tokens which are only containers, lifting their children out to flatten the tree a bit

    • removeStringLiterals

      static TokenFilter removeStringLiterals()
      Returns:
      A token filter that removes all string literal tokens

    • byName

      static TokenFilter byName(CullStrategy strategy, String... names)
      Parameters:
      strategy - The strategy to use to delete all tokens matching one of the given names
      names - The token names to process
      Returns:
      A token filter to delete tokens with the given names

    • removeUnnamed

      static TokenFilter removeUnnamed(CullStrategy strategy)
      Parameters:
      strategy - The strategy to use to delete all tokens which are unnamed
      Returns:
      A token filter to delete all unnamed tokens

    • test

      CullStrategy test(Token token)