StandardTokenizer instead.@Deprecated public final class IndicTokenizer extends CharTokenizer
AttributeSource.AttributeFactory, AttributeSource.State| Constructor and Description |
|---|
IndicTokenizer(Version matchVersion,
AttributeSource.AttributeFactory factory,
Reader input)
Deprecated.
|
IndicTokenizer(Version matchVersion,
AttributeSource source,
Reader input)
Deprecated.
|
IndicTokenizer(Version matchVersion,
Reader input)
Deprecated.
|
| Modifier and Type | Method and Description |
|---|---|
protected boolean |
isTokenChar(int c)
Deprecated.
Returns true iff a codepoint should be included in a token.
|
end, incrementToken, isTokenChar, normalize, normalize, resetclose, correctOffsetresetaddAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toStringpublic IndicTokenizer(Version matchVersion, AttributeSource.AttributeFactory factory, Reader input)
public IndicTokenizer(Version matchVersion, AttributeSource source, Reader input)
protected boolean isTokenChar(int c)
CharTokenizer
As of Lucene 3.1 the char based API (CharTokenizer.isTokenChar(char) and
CharTokenizer.normalize(char)) has been depreciated in favor of a Unicode 4.0
compatible int based API to support codepoints instead of UTF-16 code
units. Subclasses of CharTokenizer must not override the char based
methods if a Version >= 3.1 is passed to the constructor.
NOTE: This method will be marked abstract in Lucene 4.0.
isTokenChar in class CharTokenizerCopyright © 2000-2015 Apache Software Foundation. All Rights Reserved.