Package org.carrot2.text.analysis

Lexical analysis utilities.

See:
Description

Interface Summary

ITokenizer

Splits input characters into tokens representing e.g.

Class Summary

ExtendedWhitespaceTokenizer	A tokenizer separating input characters on whitespace, but capable of extracting more complex tokens, such as URLs, e-mail addresses and sentence delimiters.
TokenTypeUtils	Utility methods for working with ITokenizer attributes.

Package org.carrot2.text.analysis Description

Lexical analysis utilities.