com.aliasi.tokenizer
Interface TokenizerFactory

All Known Implementing Classes:

CharacterTokenizerFactory, IndoEuropeanTokenizerFactory, LineTokenizerFactory, NGramTokenizerFactory, RegExTokenizerFactory

public interface TokenizerFactory

A TokenizerFactory constructors tokenizers from subsequences of character arrays. Factories are typically supplied with no-argument constructors through which they may be instantiated through reflection.

Since:

LingPipe1.0

Version:

1.0

Author:

Bob Carpenter

Method Summary

Tokenizer

tokenizer(char[] ch, int start, int length) Returns a tokenizer for the specified subsequence of characters.

Method Detail

tokenizer

Tokenizer tokenizer(char[] ch,
                    int start,
                    int length)

Returns a tokenizer for the specified subsequence of characters.

Parameters:

ch - Characters to tokenize.

start - Index of first character to tokenize.

length - Number of characters to tokenize.