SinkTokenizer (Lucene 2.4.1 API)
Keywords: org.apache.lucene.analysis.SinkTokenizer class

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.apache.lucene.analysis
Class SinkTokenizer

    java.lang.Object
      org.apache.lucene.analysis.TokenStream
          org.apache.lucene.analysis.Tokenizer
              org.apache.lucene.analysis.SinkTokenizer

Direct Known Subclasses:
  DateRecognizerSinkTokenizer, TokenRangeSinkTokenizer, TokenTypeSinkTokenizer

    public class SinkTokenizer

extends Tokenizer

A SinkTokenizer can be used to cache Tokens for use in an Analyzer

See Also:
  `TeeTokenFilter`

Field Summary
`protected Iterator` | `iter`
`protected List` | `lst`

Fields inherited from class org.apache.lucene.analysis.Tokenizer
`input`

Constructor Summary
`SinkTokenizer()`
`SinkTokenizer(int initCap)`
`SinkTokenizer(List input)`

Method Summary
`void` | `add(Token t)` Override this method to cache only certain tokens, or new tokens based on the old tokens.
`void` | `close()` By default, closes the input Reader.
`List` | `getTokens()` Get the tokens in the internal List.
`Token` | `next(Token reusableToken)` Returns the next token out of the list of cached tokens
`void` | `reset()` Reset the internal data structures to the start at the front of the list of tokens.

Methods inherited from class org.apache.lucene.analysis.Tokenizer
`reset`

Methods inherited from class org.apache.lucene.analysis.TokenStream
`next`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### lst

    protected List lst

### iter

    protected Iterator iter

Constructor Detail

### SinkTokenizer

    public SinkTokenizer(List input)

### SinkTokenizer

    public SinkTokenizer()

### SinkTokenizer

    public SinkTokenizer(int initCap)

Method Detail

### getTokens

    public List getTokens()

  Get the tokens in the internal List.

  WARNING: Adding tokens to this list requires the `reset()` method to be called in order for them to be made available. Also, this Tokenizer does nothing to protect against `ConcurrentModificationException`s in the case of adds happening while `next(org.apache.lucene.analysis.Token)` is being called.

  WARNING: Since this SinkTokenizer can be reset and the cached tokens made available again, do not modify them. Modify clones instead.

  Returns:
    A List of `Token`s

### next

    public Token next(Token reusableToken)
               throws IOException

  Returns the next token out of the list of cached tokens

  Overrides:
    `next` in class `TokenStream`
  Parameters:
    `reusableToken` - a Token that may or may not be used to return; this parameter should never be null (the callee is not required to check for null before using it, but it is a good idea to assert that it is not null.)
  Returns:
    The next `Token` in the Sink.
  Throws:
    `IOException`

### add

    public void add(Token t)

  Override this method to cache only certain tokens, or new tokens based on the old tokens.

  Parameters:
    `t` - The `Token` to add to the sink

### close

    public void close()
               throws IOException

  Description copied from class: `Tokenizer`
  By default, closes the input Reader.

  Overrides:
    `close` in class `Tokenizer`
  Throws:
    `IOException`

### reset

    public void reset()
               throws IOException

  Reset the internal data structures to the start at the front of the list of tokens. Should be called if tokens were added to the list after an invocation of `next(Token)`

  Overrides:
    `reset` in class `TokenStream`
  Throws:
    `IOException`

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2000-2009 Apache Software Foundation. All Rights Reserved.