org.antlr.grammar.v2
Class AssignTokenTypesWalker

java.lang.Object
  extended by antlr.TreeParser
      extended by org.antlr.grammar.v2.AssignTokenTypesWalker
All Implemented Interfaces:
AssignTokenTypesWalkerTokenTypes
Direct Known Subclasses:
AssignTokenTypesBehavior

public class AssignTokenTypesWalker
extends antlr.TreeParser
implements AssignTokenTypesWalkerTokenTypes

[Warning: TJP says that this is probably out of date as of 11/19/2005, but since it's probably still useful, I'll leave in. Don't have energy to update at the moment.] Compute the token types for all literals and rules etc.. There are a few different cases to consider for grammar types and a few situations within. CASE 1 : pure parser grammar a) Any reference to a token gets a token type. b) The tokens section may alias a token name to a string or char CASE 2 : pure lexer grammar a) Import token vocabulary if available. Set token types for any new tokens to values above last imported token type b) token rule definitions get token types if not already defined c) literals do NOT get token types CASE 3 : merged parser / lexer grammar a) Any char or string literal gets a token type in a parser rule b) Any reference to a token gets a token type if not referencing a fragment lexer rule c) The tokens section may alias a token name to a string or char which must add a rule to the lexer d) token rule definitions get token types if not already defined e) token rule definitions may also alias a token name to a literal. E.g., Rule 'FOR : "for";' will alias FOR to "for" in the sense that references to either in the parser grammar will yield the token type What this pass does: 0. Collects basic info about the grammar like grammar name and type; Oh, I have go get the options in case they affect the token types. E.g., tokenVocab option. Imports any token vocab name/type pairs into a local hashtable. 1. Finds a list of all literals and token names. 2. Finds a list of all token name rule definitions; no token rules implies pure parser. 3. Finds a list of all simple token rule defs of form " : ;" and aliases them. 4. Walks token names table and assign types to any unassigned 5. Walks aliases and assign types to referenced literals 6. Walks literals, assigning types if untyped 4. Informs the Grammar object of the type definitions such as: g.defineToken(, ttype); g.defineToken(, ttype); g.defineToken(, ttype); where some of the ttype values will be the same for aliases tokens.


Field Summary
static java.lang.String[] _tokenNames
           
protected static GrammarAST charAlias
           
protected static GrammarAST charAlias2
           
protected  java.lang.String currentRuleName
           
protected  Grammar grammar
           
protected static GrammarAST stringAlias
           
protected static GrammarAST stringAlias2
           
 
Fields inherited from class antlr.TreeParser
_retTree, astFactory, ASTNULL, inputState, returnAST, tokenNames, traceDepth
 
Fields inherited from interface org.antlr.grammar.v2.AssignTokenTypesWalkerTokenTypes
ACTION, ACTION_CHAR_LITERAL, ACTION_ESC, ACTION_STRING_LITERAL, ALT, AMPERSAND, ARG, ARG_ACTION, ARGLIST, ASSIGN, BACKTRACK_SEMPRED, BANG, BLOCK, CHAR_LITERAL, CHAR_RANGE, CHARSET, CLOSE_ELEMENT_OPTION, CLOSURE, COLON, COMBINED_GRAMMAR, COMMA, COMMENT, DIGIT, DOC_COMMENT, DOLLAR, DOT, DOUBLE_ANGLE_STRING_LITERAL, DOUBLE_QUOTE_STRING_LITERAL, EOA, EOB, EOF, EOR, EPSILON, ESC, ETC, FORCED_ACTION, FRAGMENT, GATED_SEMPRED, ID, IMPLIES, IMPORT, INITACTION, INT, INTERNAL_RULE_REF, LABEL, LEXER, LEXER_GRAMMAR, LITERAL_catch, LITERAL_finally, LITERAL_grammar, LITERAL_lexer, LITERAL_private, LITERAL_protected, LITERAL_public, LITERAL_returns, LITERAL_throws, LITERAL_tree, LPAREN, ML_COMMENT, NESTED_ACTION, NESTED_ARG_ACTION, NOT, NULL_TREE_LOOKAHEAD, OPEN_ELEMENT_OPTION, OPTIONAL, OPTIONS, OR, PARSER, PARSER_GRAMMAR, PLUS, PLUS_ASSIGN, POSITIVE_CLOSURE, QUESTION, RANGE, RCURLY, RET, REWRITE, ROOT, RPAREN, RULE, RULE_REF, SCOPE, SEMI, SEMPRED, SL_COMMENT, SRC, STAR, STRAY_BRACKET, STRING_LITERAL, SYN_SEMPRED, SYNPRED, TEMPLATE, TOKEN_REF, TOKENS, TREE_BEGIN, TREE_GRAMMAR, WILDCARD, WS, WS_LOOP, WS_OPT, XDIGIT
 
Constructor Summary
AssignTokenTypesWalker()
           
 
Method Summary
protected  void alias(GrammarAST t, GrammarAST s)
           
protected  void aliasTokenIDsAndLiterals(Grammar root)
           
 void alternative(antlr.collections.AST _t)
           
protected  void assignStringTypes(Grammar root)
           
protected  void assignTokenIDTypes(Grammar root)
           
 void ast_suffix(antlr.collections.AST _t)
           
 void atom(antlr.collections.AST _t)
           
 void attrScope(antlr.collections.AST _t)
           
 void block(antlr.collections.AST _t)
           
 void charSet(antlr.collections.AST _t)
           
 void charSetElement(antlr.collections.AST _t)
           
protected  void defineStringLiteralsFromDelegates()
           
protected  void defineTokenNamesAndLiteralsInGrammar(Grammar root)
           
 void defineTokens(Grammar root)
           
 void delegateGrammars(antlr.collections.AST _t)
           
 void ebnf(antlr.collections.AST _t)
           
 void element(antlr.collections.AST _t)
           
 void exceptionGroup(antlr.collections.AST _t)
           
 void exceptionHandler(antlr.collections.AST _t)
           
 void finallyClause(antlr.collections.AST _t)
           
 void grammar(antlr.collections.AST _t, Grammar g)
           
 void grammarSpec(antlr.collections.AST _t)
           
protected  void init(Grammar root)
           
protected  void initASTPatterns()
           
 void modifier(antlr.collections.AST _t)
           
 void option(antlr.collections.AST _t, java.util.Map opts)
           
 java.util.Map optionsSpec(antlr.collections.AST _t)
           
 java.lang.Object optionValue(antlr.collections.AST _t)
           
 void reportError(antlr.RecognitionException ex)
           
 void rewrite(antlr.collections.AST _t)
           
 void rule(antlr.collections.AST _t)
           
 void rules(antlr.collections.AST _t)
           
 void ruleScopeSpec(antlr.collections.AST _t)
           
 void tokenSpec(antlr.collections.AST _t)
           
 void tokensSpec(antlr.collections.AST _t)
           
protected  void trackString(GrammarAST t)
           
protected  void trackToken(GrammarAST t)
           
protected  void trackTokenRule(GrammarAST t, GrammarAST modifier, GrammarAST block)
           
 void tree(antlr.collections.AST _t)
           
 
Methods inherited from class antlr.TreeParser
getAST, getASTFactory, getTokenName, getTokenNames, match, match, matchNot, panic, reportError, reportWarning, setASTFactory, setASTNodeClass, setASTNodeType, traceIn, traceIndent, traceOut
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

grammar

protected Grammar grammar

currentRuleName

protected java.lang.String currentRuleName

stringAlias

protected static GrammarAST stringAlias

charAlias

protected static GrammarAST charAlias

stringAlias2

protected static GrammarAST stringAlias2

charAlias2

protected static GrammarAST charAlias2

_tokenNames

public static final java.lang.String[] _tokenNames
Constructor Detail

AssignTokenTypesWalker

public AssignTokenTypesWalker()
Method Detail

reportError

public void reportError(antlr.RecognitionException ex)
Overrides:
reportError in class antlr.TreeParser

initASTPatterns

protected void initASTPatterns()

trackString

protected void trackString(GrammarAST t)

trackToken

protected void trackToken(GrammarAST t)

trackTokenRule

protected void trackTokenRule(GrammarAST t,
                              GrammarAST modifier,
                              GrammarAST block)

alias

protected void alias(GrammarAST t,
                     GrammarAST s)

defineTokens

public void defineTokens(Grammar root)

defineStringLiteralsFromDelegates

protected void defineStringLiteralsFromDelegates()

assignStringTypes

protected void assignStringTypes(Grammar root)

aliasTokenIDsAndLiterals

protected void aliasTokenIDsAndLiterals(Grammar root)

assignTokenIDTypes

protected void assignTokenIDTypes(Grammar root)

defineTokenNamesAndLiteralsInGrammar

protected void defineTokenNamesAndLiteralsInGrammar(Grammar root)

init

protected void init(Grammar root)

grammar

public final void grammar(antlr.collections.AST _t,
                          Grammar g)
                   throws antlr.RecognitionException
Throws:
antlr.RecognitionException

grammarSpec

public final void grammarSpec(antlr.collections.AST _t)
                       throws antlr.RecognitionException
Throws:
antlr.RecognitionException

optionsSpec

public final java.util.Map optionsSpec(antlr.collections.AST _t)
                                throws antlr.RecognitionException
Throws:
antlr.RecognitionException

delegateGrammars

public final void delegateGrammars(antlr.collections.AST _t)
                            throws antlr.RecognitionException
Throws:
antlr.RecognitionException

tokensSpec

public final void tokensSpec(antlr.collections.AST _t)
                      throws antlr.RecognitionException
Throws:
antlr.RecognitionException

attrScope

public final void attrScope(antlr.collections.AST _t)
                     throws antlr.RecognitionException
Throws:
antlr.RecognitionException

rules

public final void rules(antlr.collections.AST _t)
                 throws antlr.RecognitionException
Throws:
antlr.RecognitionException

option

public final void option(antlr.collections.AST _t,
                         java.util.Map opts)
                  throws antlr.RecognitionException
Throws:
antlr.RecognitionException

optionValue

public final java.lang.Object optionValue(antlr.collections.AST _t)
                                   throws antlr.RecognitionException
Throws:
antlr.RecognitionException

charSet

public final void charSet(antlr.collections.AST _t)
                   throws antlr.RecognitionException
Throws:
antlr.RecognitionException

charSetElement

public final void charSetElement(antlr.collections.AST _t)
                          throws antlr.RecognitionException
Throws:
antlr.RecognitionException

tokenSpec

public final void tokenSpec(antlr.collections.AST _t)
                     throws antlr.RecognitionException
Throws:
antlr.RecognitionException

rule

public final void rule(antlr.collections.AST _t)
                throws antlr.RecognitionException
Throws:
antlr.RecognitionException

modifier

public final void modifier(antlr.collections.AST _t)
                    throws antlr.RecognitionException
Throws:
antlr.RecognitionException

ruleScopeSpec

public final void ruleScopeSpec(antlr.collections.AST _t)
                         throws antlr.RecognitionException
Throws:
antlr.RecognitionException

block

public final void block(antlr.collections.AST _t)
                 throws antlr.RecognitionException
Throws:
antlr.RecognitionException

exceptionGroup

public final void exceptionGroup(antlr.collections.AST _t)
                          throws antlr.RecognitionException
Throws:
antlr.RecognitionException

alternative

public final void alternative(antlr.collections.AST _t)
                       throws antlr.RecognitionException
Throws:
antlr.RecognitionException

rewrite

public final void rewrite(antlr.collections.AST _t)
                   throws antlr.RecognitionException
Throws:
antlr.RecognitionException

element

public final void element(antlr.collections.AST _t)
                   throws antlr.RecognitionException
Throws:
antlr.RecognitionException

exceptionHandler

public final void exceptionHandler(antlr.collections.AST _t)
                            throws antlr.RecognitionException
Throws:
antlr.RecognitionException

finallyClause

public final void finallyClause(antlr.collections.AST _t)
                         throws antlr.RecognitionException
Throws:
antlr.RecognitionException

atom

public final void atom(antlr.collections.AST _t)
                throws antlr.RecognitionException
Throws:
antlr.RecognitionException

ebnf

public final void ebnf(antlr.collections.AST _t)
                throws antlr.RecognitionException
Throws:
antlr.RecognitionException

tree

public final void tree(antlr.collections.AST _t)
                throws antlr.RecognitionException
Throws:
antlr.RecognitionException

ast_suffix

public final void ast_suffix(antlr.collections.AST _t)
                      throws antlr.RecognitionException
Throws:
antlr.RecognitionException


Copyright © 2013. All Rights Reserved.