org.antlr.grammar.v2
Class AssignTokenTypesWalker
java.lang.Object
antlr.TreeParser
org.antlr.grammar.v2.AssignTokenTypesWalker
- All Implemented Interfaces:
- AssignTokenTypesWalkerTokenTypes
- Direct Known Subclasses:
- AssignTokenTypesBehavior
public class AssignTokenTypesWalker
- extends antlr.TreeParser
- implements AssignTokenTypesWalkerTokenTypes
[Warning: TJP says that this is probably out of date as of 11/19/2005,
but since it's probably still useful, I'll leave in. Don't have energy
to update at the moment.]
Compute the token types for all literals and rules etc.. There are
a few different cases to consider for grammar types and a few situations
within.
CASE 1 : pure parser grammar
a) Any reference to a token gets a token type.
b) The tokens section may alias a token name to a string or char
CASE 2 : pure lexer grammar
a) Import token vocabulary if available. Set token types for any new tokens
to values above last imported token type
b) token rule definitions get token types if not already defined
c) literals do NOT get token types
CASE 3 : merged parser / lexer grammar
a) Any char or string literal gets a token type in a parser rule
b) Any reference to a token gets a token type if not referencing
a fragment lexer rule
c) The tokens section may alias a token name to a string or char
which must add a rule to the lexer
d) token rule definitions get token types if not already defined
e) token rule definitions may also alias a token name to a literal.
E.g., Rule 'FOR : "for";' will alias FOR to "for" in the sense that
references to either in the parser grammar will yield the token type
What this pass does:
0. Collects basic info about the grammar like grammar name and type;
Oh, I have go get the options in case they affect the token types.
E.g., tokenVocab option.
Imports any token vocab name/type pairs into a local hashtable.
1. Finds a list of all literals and token names.
2. Finds a list of all token name rule definitions;
no token rules implies pure parser.
3. Finds a list of all simple token rule defs of form " : ;"
and aliases them.
4. Walks token names table and assign types to any unassigned
5. Walks aliases and assign types to referenced literals
6. Walks literals, assigning types if untyped
4. Informs the Grammar object of the type definitions such as:
g.defineToken(, ttype);
g.defineToken(, ttype);
g.defineToken(, ttype);
where some of the ttype values will be the same for aliases tokens.
Fields inherited from class antlr.TreeParser |
_retTree, astFactory, ASTNULL, inputState, returnAST, tokenNames, traceDepth |
Fields inherited from interface org.antlr.grammar.v2.AssignTokenTypesWalkerTokenTypes |
ACTION, ACTION_CHAR_LITERAL, ACTION_ESC, ACTION_STRING_LITERAL, ALT, AMPERSAND, ARG, ARG_ACTION, ARGLIST, ASSIGN, BACKTRACK_SEMPRED, BANG, BLOCK, CHAR_LITERAL, CHAR_RANGE, CHARSET, CLOSE_ELEMENT_OPTION, CLOSURE, COLON, COMBINED_GRAMMAR, COMMA, COMMENT, DIGIT, DOC_COMMENT, DOLLAR, DOT, DOUBLE_ANGLE_STRING_LITERAL, DOUBLE_QUOTE_STRING_LITERAL, EOA, EOB, EOF, EOR, EPSILON, ESC, ETC, FORCED_ACTION, FRAGMENT, GATED_SEMPRED, ID, IMPLIES, IMPORT, INITACTION, INT, INTERNAL_RULE_REF, LABEL, LEXER, LEXER_GRAMMAR, LITERAL_catch, LITERAL_finally, LITERAL_grammar, LITERAL_lexer, LITERAL_private, LITERAL_protected, LITERAL_public, LITERAL_returns, LITERAL_throws, LITERAL_tree, LPAREN, ML_COMMENT, NESTED_ACTION, NESTED_ARG_ACTION, NOT, NULL_TREE_LOOKAHEAD, OPEN_ELEMENT_OPTION, OPTIONAL, OPTIONS, OR, PARSER, PARSER_GRAMMAR, PLUS, PLUS_ASSIGN, POSITIVE_CLOSURE, QUESTION, RANGE, RCURLY, RET, REWRITE, ROOT, RPAREN, RULE, RULE_REF, SCOPE, SEMI, SEMPRED, SL_COMMENT, SRC, STAR, STRAY_BRACKET, STRING_LITERAL, SYN_SEMPRED, SYNPRED, TEMPLATE, TOKEN_REF, TOKENS, TREE_BEGIN, TREE_GRAMMAR, WILDCARD, WS, WS_LOOP, WS_OPT, XDIGIT |
Methods inherited from class antlr.TreeParser |
getAST, getASTFactory, getTokenName, getTokenNames, match, match, matchNot, panic, reportError, reportWarning, setASTFactory, setASTNodeClass, setASTNodeType, traceIn, traceIndent, traceOut |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
grammar
protected Grammar grammar
currentRuleName
protected java.lang.String currentRuleName
stringAlias
protected static GrammarAST stringAlias
charAlias
protected static GrammarAST charAlias
stringAlias2
protected static GrammarAST stringAlias2
charAlias2
protected static GrammarAST charAlias2
_tokenNames
public static final java.lang.String[] _tokenNames
AssignTokenTypesWalker
public AssignTokenTypesWalker()
reportError
public void reportError(antlr.RecognitionException ex)
- Overrides:
reportError
in class antlr.TreeParser
initASTPatterns
protected void initASTPatterns()
trackString
protected void trackString(GrammarAST t)
trackToken
protected void trackToken(GrammarAST t)
trackTokenRule
protected void trackTokenRule(GrammarAST t,
GrammarAST modifier,
GrammarAST block)
alias
protected void alias(GrammarAST t,
GrammarAST s)
defineTokens
public void defineTokens(Grammar root)
defineStringLiteralsFromDelegates
protected void defineStringLiteralsFromDelegates()
assignStringTypes
protected void assignStringTypes(Grammar root)
aliasTokenIDsAndLiterals
protected void aliasTokenIDsAndLiterals(Grammar root)
assignTokenIDTypes
protected void assignTokenIDTypes(Grammar root)
defineTokenNamesAndLiteralsInGrammar
protected void defineTokenNamesAndLiteralsInGrammar(Grammar root)
init
protected void init(Grammar root)
grammar
public final void grammar(antlr.collections.AST _t,
Grammar g)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
grammarSpec
public final void grammarSpec(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
optionsSpec
public final java.util.Map optionsSpec(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
delegateGrammars
public final void delegateGrammars(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
tokensSpec
public final void tokensSpec(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
attrScope
public final void attrScope(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
rules
public final void rules(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
option
public final void option(antlr.collections.AST _t,
java.util.Map opts)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
optionValue
public final java.lang.Object optionValue(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
charSet
public final void charSet(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
charSetElement
public final void charSetElement(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
tokenSpec
public final void tokenSpec(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
rule
public final void rule(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
modifier
public final void modifier(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
ruleScopeSpec
public final void ruleScopeSpec(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
block
public final void block(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
exceptionGroup
public final void exceptionGroup(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
alternative
public final void alternative(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
rewrite
public final void rewrite(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
element
public final void element(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
exceptionHandler
public final void exceptionHandler(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
finallyClause
public final void finallyClause(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
atom
public final void atom(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
ebnf
public final void ebnf(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
tree
public final void tree(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
ast_suffix
public final void ast_suffix(antlr.collections.AST _t)
throws antlr.RecognitionException
- Throws:
antlr.RecognitionException
Copyright © 2013. All Rights Reserved.