StandardTokenizer instead, which has the same functionality.
This filter will be removed in Lucene 5.0@Deprecated public final class ChineseTokenizer extends org.apache.lucene.analysis.Tokenizer
The difference between ChineseTokenizer and CJKTokenizer is that they have different token parsing logic.
For example, if the Chinese text "C1C2C3C4" is to be indexed:
Therefore the index created by CJKTokenizer is much larger.
The problem is that when searching for C1, C1C2, C1C3, C4C2, C1C2C3 ... the ChineseTokenizer works, but the CJKTokenizer will not work.
| Constructor and Description |
|---|
ChineseTokenizer(org.apache.lucene.util.AttributeSource.AttributeFactory factory,
Reader in)
Deprecated.
|
ChineseTokenizer(org.apache.lucene.util.AttributeSource source,
Reader in)
Deprecated.
|
ChineseTokenizer(Reader in)
Deprecated.
|
| Modifier and Type | Method and Description |
|---|---|
void |
end()
Deprecated.
|
boolean |
incrementToken()
Deprecated.
|
void |
reset()
Deprecated.
|
void |
reset(Reader input)
Deprecated.
|
addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, restoreState, toStringpublic ChineseTokenizer(Reader in)
public ChineseTokenizer(org.apache.lucene.util.AttributeSource source,
Reader in)
public ChineseTokenizer(org.apache.lucene.util.AttributeSource.AttributeFactory factory,
Reader in)
public boolean incrementToken()
throws IOException
incrementToken in class org.apache.lucene.analysis.TokenStreamIOExceptionpublic final void end()
end in class org.apache.lucene.analysis.TokenStreampublic void reset()
throws IOException
reset in class org.apache.lucene.analysis.TokenStreamIOExceptionpublic void reset(Reader input) throws IOException
reset in class org.apache.lucene.analysis.TokenizerIOExceptionCopyright © 2000-2015 Apache Software Foundation. All Rights Reserved.