Uses of Class
org.apache.lucene.analysis.TokenFilter
-
Packages that use TokenFilter Package Description org.apache.lucene.analysis API and code to convert text into indexable/searchable tokens.org.apache.lucene.analysis.standard Standards-based analyzers implemented with JFlex.org.apache.lucene.collation CollationKeyFilter
converts each token into its binaryCollationKey
using the providedCollator
, and then encode theCollationKey
as a String usingIndexableBinaryStringTools
, to allow it to be stored as an index term. -
-
Uses of TokenFilter in org.apache.lucene.analysis
Subclasses of TokenFilter in org.apache.lucene.analysis Modifier and Type Class Description class
ASCIIFoldingFilter
This class converts alphabetic, numeric, and symbolic Unicode characters which are not in the first 127 ASCII characters (the "Basic Latin" Unicode block) into their ASCII equivalents, if one exists.class
CachingTokenFilter
This class can be used if the token attributes of a TokenStream are intended to be consumed more than once.class
FilteringTokenFilter
Abstract base class for TokenFilters that may remove tokens.class
ISOLatin1AccentFilter
Deprecated.If you build a new index, useASCIIFoldingFilter
which covers a superset of Latin 1.class
KeywordMarkerFilter
Marks terms as keywords via theKeywordAttribute
.class
LengthFilter
Removes words that are too long or too short from the stream.class
LimitTokenCountFilter
This TokenFilter limits the number of tokens while indexing.class
LowerCaseFilter
Normalizes token text to lower case.class
PorterStemFilter
Transforms the token stream as per the Porter stemming algorithm.class
StopFilter
Removes stop words from a token stream.class
TeeSinkTokenFilter
This TokenFilter provides the ability to set aside attribute states that have already been analyzed.class
TypeTokenFilter
Removes tokens whose types appear in a set of blocked types from a token stream. -
Uses of TokenFilter in org.apache.lucene.analysis.standard
Subclasses of TokenFilter in org.apache.lucene.analysis.standard Modifier and Type Class Description class
ClassicFilter
Normalizes tokens extracted withClassicTokenizer
.class
StandardFilter
Normalizes tokens extracted withStandardTokenizer
. -
Uses of TokenFilter in org.apache.lucene.collation
Subclasses of TokenFilter in org.apache.lucene.collation Modifier and Type Class Description class
CollationKeyFilter
Converts each token into itsCollationKey
, and then encodes the CollationKey withIndexableBinaryStringTools
, to allow it to be stored as an index term.
-