opennlp.tools.tokenize
Class DefaultTokenContextGenerator

java.lang.Object
  opennlp.tools.tokenize.DefaultTokenContextGenerator

All Implemented Interfaces:

TokenContextGenerator

public class DefaultTokenContextGenerator
extends java.lang.Object
implements TokenContextGenerator

Generate events for maxent decisions for tokenization.

Version:

$Revision: 1.1 $, $Date: 2008/03/05 16:45:13 $

Author:

Jason Baldridge

Field Summary

static java.lang.String	NO_SPLIT Constant indicates no token split.
static java.lang.String	SPLIT Constant indicates a token split.

Constructor Summary

DefaultTokenContextGenerator()

Method Summary

java.lang.String[]

getContext(java.lang.String sentence, int index) Returns an array of features for the specified sentence string at the specified index.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SPLIT

public static final java.lang.String SPLIT

Constant indicates a token split.

See Also:

Constant Field Values

NO_SPLIT

public static final java.lang.String NO_SPLIT

Constant indicates no token split.

See Also:

Constant Field Values

Constructor Detail

DefaultTokenContextGenerator

public DefaultTokenContextGenerator()

Method Detail

getContext

public java.lang.String[] getContext(java.lang.String sentence,
                                     int index)

Description copied from interface: TokenContextGenerator

Returns an array of features for the specified sentence string at the specified index.

Specified by:

getContext in interface TokenContextGenerator

Parameters:

sentence - The string for a sentence.

index - The index to consider splitting as a token.

Returns:

an array of features for the specified sentence string at the specified index.