antlr

Class CharScanner

Implemented Interfaces:
TokenStream
Known Direct Subclasses:
ActionLexer, ActionLexer, ActionLexer, ActionLexer, ANTLRLexer, ANTLRTokdefLexer, CodeLexer, DebuggingCharScanner, PreprocessorLexer

public abstract class CharScanner
extends Object
implements TokenStream

Field Summary

static char
EOF_CHAR
(package private) static char
NO_CHAR
protected Token
_returnToken
protected boolean
caseSensitive
protected boolean
caseSensitiveLiterals
protected boolean
commitToPath
Used during filter mode to indicate that path is desired.
protected ANTLRHashString
hashString
protected LexerSharedInputState
inputState
protected Hashtable
literals
protected boolean
saveConsumedInput
protected int
tabsize
Tab chars are handled by tab() according to this value; override method to do anything weird with tabs.
protected ANTLRStringBuffer
text
protected Class
tokenObjectClass
protected int
traceDepth
Used to keep track of indentdepth for traceIn/Out

Constructor Summary

CharScanner()
CharScanner(InputBuffer cb)
CharScanner(LexerSharedInputState sharedState)

Method Summary

char
LA(int i)
void
append(String s)
void
append(char c)
void
commit()
void
consume()
void
consumeUntil(BitSet set)
Consume chars until one matches the given set
void
consumeUntil(int c)
Consume chars until one matches the given char
boolean
getCaseSensitive()
boolean
getCaseSensitiveLiterals()
int
getColumn()
boolean
getCommitToPath()
String
getFilename()
InputBuffer
getInputBuffer()
LexerSharedInputState
getInputState()
int
getLine()
int
getTabSize()
String
getText()
return a copy of the current text buffer
Token
getTokenObject()
protected Token
makeToken(int t)
int
mark()
void
match(String s)
void
match(BitSet b)
void
match(char c)
void
matchNot(char c)
void
matchRange(char c1, char c2)
void
newline()
void
panic()
void
panic(String s)
This method is executed by ANTLR internally when it detected an illegal state that cannot be recovered from.
void
reportError(String s)
Parser error-reporting function can be overridden in subclass
void
reportError(RecognitionException ex)
Parser error-reporting function can be overridden in subclass
void
reportWarning(String s)
Parser warning-reporting function can be overridden in subclass
void
resetText()
void
rewind(int pos)
void
setCaseSensitive(boolean t)
void
setColumn(int c)
void
setCommitToPath(boolean commit)
void
setFilename(String f)
void
setInputState(LexerSharedInputState state)
void
setLine(int line)
void
setTabSize(int size)
void
setText(String s)
void
setTokenObjectClass(String cl)
void
tab()
advance the current column number by an appropriate amount according to tab size.
int
testLiteralsTable(String text, int ttype)
Test the text passed in against the literals table Override this method to perform a different literals test This is used primarily when you want to test a portion of a token.
int
testLiteralsTable(int ttype)
char
toLower(char c)
void
traceIn(String rname)
void
traceIndent()
void
traceOut(String rname)
void
uponEOF()
This method is called by YourLexer.nextToken() when the lexer has hit EOF condition.

Field Details

EOF_CHAR

public static final char EOF_CHAR
Field Value:
'\uffff'

NO_CHAR

(package private) static final char NO_CHAR
Field Value:
'\u0000'

_returnToken

protected Token _returnToken

caseSensitive

protected boolean caseSensitive

caseSensitiveLiterals

protected boolean caseSensitiveLiterals

commitToPath

protected boolean commitToPath
Used during filter mode to indicate that path is desired. A subsequent scan error will report an error as usual if acceptPath=true;

hashString

protected ANTLRHashString hashString

inputState

protected LexerSharedInputState inputState

literals

protected Hashtable literals

saveConsumedInput

protected boolean saveConsumedInput

tabsize

protected int tabsize
Tab chars are handled by tab() according to this value; override method to do anything weird with tabs.

text

protected ANTLRStringBuffer text

tokenObjectClass

protected Class tokenObjectClass

traceDepth

protected int traceDepth
Used to keep track of indentdepth for traceIn/Out

Constructor Details

CharScanner

public CharScanner()

CharScanner

public CharScanner(InputBuffer cb)

CharScanner

public CharScanner(LexerSharedInputState sharedState)

Method Details

LA

public char LA(int i)
            throws CharStreamException

append

public void append(String s)

append

public void append(char c)

commit

public void commit()

consume

public void consume()
            throws CharStreamException

consumeUntil

public void consumeUntil(BitSet set)
            throws CharStreamException
Consume chars until one matches the given set

consumeUntil

public void consumeUntil(int c)
            throws CharStreamException
Consume chars until one matches the given char

getCaseSensitive

public boolean getCaseSensitive()

getCaseSensitiveLiterals

public final boolean getCaseSensitiveLiterals()

getColumn

public int getColumn()

getCommitToPath

public boolean getCommitToPath()

getFilename

public String getFilename()

getInputBuffer

public InputBuffer getInputBuffer()

getInputState

public LexerSharedInputState getInputState()

getLine

public int getLine()

getTabSize

public int getTabSize()

getText

public String getText()
return a copy of the current text buffer

getTokenObject

public Token getTokenObject()

makeToken

protected Token makeToken(int t)

mark

public int mark()

match

public void match(String s)
            throws MismatchedCharException,
                   CharStreamException

match

public void match(BitSet b)
            throws MismatchedCharException,
                   CharStreamException

match

public void match(char c)
            throws MismatchedCharException,
                   CharStreamException

matchNot

public void matchNot(char c)
            throws MismatchedCharException,
                   CharStreamException

matchRange

public void matchRange(char c1,
                       char c2)
            throws MismatchedCharException,
                   CharStreamException

newline

public void newline()

panic

public void panic()

panic

public void panic(String s)
This method is executed by ANTLR internally when it detected an illegal state that cannot be recovered from. The default implementation of this method calls System.exit(int) and writes directly to System.err) , which is usually not appropriate when a translator is embedded into a larger application. It is highly recommended that this method be overridden to handle the error in a way appropriate for your application (e.g. throw an unchecked exception).

reportError

public void reportError(String s)
Parser error-reporting function can be overridden in subclass

reportError

public void reportError(RecognitionException ex)
Parser error-reporting function can be overridden in subclass

reportWarning

public void reportWarning(String s)
Parser warning-reporting function can be overridden in subclass

resetText

public void resetText()

rewind

public void rewind(int pos)

setCaseSensitive

public void setCaseSensitive(boolean t)

setColumn

public void setColumn(int c)

setCommitToPath

public void setCommitToPath(boolean commit)

setFilename

public void setFilename(String f)

setInputState

public void setInputState(LexerSharedInputState state)

setLine

public void setLine(int line)

setTabSize

public void setTabSize(int size)

setText

public void setText(String s)

setTokenObjectClass

public void setTokenObjectClass(String cl)

tab

public void tab()
advance the current column number by an appropriate amount according to tab size. This method is called from consume().

testLiteralsTable

public int testLiteralsTable(String text,
                             int ttype)
Test the text passed in against the literals table Override this method to perform a different literals test This is used primarily when you want to test a portion of a token.

testLiteralsTable

public int testLiteralsTable(int ttype)

toLower

public char toLower(char c)

traceIn

public void traceIn(String rname)
            throws CharStreamException

traceIndent

public void traceIndent()

traceOut

public void traceOut(String rname)
            throws CharStreamException

uponEOF

public void uponEOF()
            throws TokenStreamException,
                   CharStreamException
This method is called by YourLexer.nextToken() when the lexer has hit EOF condition. EOF is NOT a character. This method is not called if EOF is reached during syntactic predicate evaluation or during evaluation of normal lexical rules, which presumably would be an IOException. This traps the "normal" EOF condition. uponEOF() is called after the complete evaluation of the previous token and only if your parser asks for another token beyond that last non-EOF token. You might want to throw token or char stream exceptions like: "Heh, premature eof" or a retry stream exception ("I found the end of this file, go back to referencing file").