StandardAnalyzer (Lucene 2.4.1 API)
Keywords: org.apache.lucene.analysis.standard.StandardAnalyzer class

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## org.apache.lucene.analysis.standard
Class StandardAnalyzer

    java.lang.Object
      org.apache.lucene.analysis.Analyzer
          org.apache.lucene.analysis.standard.StandardAnalyzer

    public class StandardAnalyzer

extends Analyzer

Filters `StandardTokenizer` with `StandardFilter`, `LowerCaseFilter` and `StopFilter`, using a list of English stop words.

Version:
  $Id: StandardAnalyzer.java 692634 2008-09-06 10:58:33Z mikemccand $

Field Summary
`static int` | `DEFAULT_MAX_TOKEN_LENGTH` Default maximum allowed token length
`static String[]` | `STOP_WORDS` An array containing some common English words that are usually not useful for searching.

Constructor Summary
`StandardAnalyzer()` Builds an analyzer with the default stop words (`STOP_WORDS`).
`StandardAnalyzer(boolean replaceInvalidAcronym)` Deprecated. Remove in 3.X and make true the only valid value
`StandardAnalyzer(File stopwords)` Builds an analyzer with the stop words from the given file.
`StandardAnalyzer(File stopwords, boolean replaceInvalidAcronym)` Deprecated. Remove in 3.X and make true the only valid value
`StandardAnalyzer(Reader stopwords)` Builds an analyzer with the stop words from the given reader.
`StandardAnalyzer(Reader stopwords, boolean replaceInvalidAcronym)` Deprecated. Remove in 3.X and make true the only valid value
`StandardAnalyzer(Set stopWords)` Builds an analyzer with the given stop words.
`StandardAnalyzer(Set stopwords, boolean replaceInvalidAcronym)` Deprecated. Remove in 3.X and make true the only valid value
`StandardAnalyzer(String[] stopWords)` Builds an analyzer with the given stop words.
`StandardAnalyzer(String[] stopwords, boolean replaceInvalidAcronym)` Deprecated. Remove in 3.X and make true the only valid value

Method Summary
`static boolean` | `getDefaultReplaceInvalidAcronym()` Deprecated. This will be removed (hardwired to true) in 3.0
`int` | `getMaxTokenLength()`
`boolean` | `isReplaceInvalidAcronym()` Deprecated. This will be removed (hardwired to true) in 3.0
`TokenStream` | `reusableTokenStream(String fieldName, Reader reader)` Creates a TokenStream that is allowed to be re-used from the previous time that the same thread called this method.
`static void` | `setDefaultReplaceInvalidAcronym(boolean replaceInvalidAcronym)` Deprecated. This will be removed (hardwired to true) in 3.0
`void` | `setMaxTokenLength(int length)` Set maximum allowed token length.
`void` | `setReplaceInvalidAcronym(boolean replaceInvalidAcronym)` Deprecated. This will be removed (hardwired to true) in 3.0
`TokenStream` | `tokenStream(String fieldName, Reader reader)` Constructs a `StandardTokenizer` filtered by a `StandardFilter`, a `LowerCaseFilter` and a `StopFilter`.

Methods inherited from class org.apache.lucene.analysis.Analyzer
`close, getPositionIncrementGap, getPreviousTokenStream, setPreviousTokenStream`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### STOP_WORDS

    public static final String[] STOP_WORDS

  An array containing some common English words that are usually not useful for searching.

### DEFAULT_MAX_TOKEN_LENGTH

    public static final int DEFAULT_MAX_TOKEN_LENGTH

  Default maximum allowed token length

  See Also:
    Constant Field Values

Constructor Detail

### StandardAnalyzer

    public StandardAnalyzer()

  Builds an analyzer with the default stop words (`STOP_WORDS`).

### StandardAnalyzer

    public StandardAnalyzer(Set stopWords)

  Builds an analyzer with the given stop words.

### StandardAnalyzer

    public StandardAnalyzer(String[] stopWords)

  Builds an analyzer with the given stop words.

### StandardAnalyzer

    public StandardAnalyzer(File stopwords)
                     throws IOException

  Builds an analyzer with the stop words from the given file.

  Throws:
    `IOException`
  See Also:
    `WordlistLoader.getWordSet(File)`

### StandardAnalyzer

    public StandardAnalyzer(Reader stopwords)
                     throws IOException

  Builds an analyzer with the stop words from the given reader.

  Throws:
    `IOException`
  See Also:
    `WordlistLoader.getWordSet(Reader)`

### StandardAnalyzer

    public StandardAnalyzer(boolean replaceInvalidAcronym)

  Deprecated. Remove in 3.X and make true the only valid value

  Parameters:
    `replaceInvalidAcronym` - Set to true if this analyzer should replace mischaracterized acronyms in the StandardTokenizer See https://issues.apache.org/jira/browse/LUCENE-1068

### StandardAnalyzer

    public StandardAnalyzer(Reader stopwords,
                            boolean replaceInvalidAcronym)
                     throws IOException

  Deprecated. Remove in 3.X and make true the only valid value

  Parameters:
    `stopwords` - The stopwords to use
    `replaceInvalidAcronym` - Set to true if this analyzer should replace mischaracterized acronyms in the StandardTokenizer See https://issues.apache.org/jira/browse/LUCENE-1068
  Throws:
    `IOException`

### StandardAnalyzer

    public StandardAnalyzer(File stopwords,
                            boolean replaceInvalidAcronym)
                     throws IOException

  Deprecated. Remove in 3.X and make true the only valid value

  Parameters:
    `stopwords` - The stopwords to use
    `replaceInvalidAcronym` - Set to true if this analyzer should replace mischaracterized acronyms in the StandardTokenizer See https://issues.apache.org/jira/browse/LUCENE-1068
  Throws:
    `IOException`

### StandardAnalyzer

    public StandardAnalyzer(String[] stopwords,
                            boolean replaceInvalidAcronym)
                     throws IOException

  Deprecated. Remove in 3.X and make true the only valid value

  Parameters:
    `stopwords` - The stopwords to use
    `replaceInvalidAcronym` - Set to true if this analyzer should replace mischaracterized acronyms in the StandardTokenizer See https://issues.apache.org/jira/browse/LUCENE-1068
  Throws:
    `IOException`

### StandardAnalyzer

    public StandardAnalyzer(Set stopwords,
                            boolean replaceInvalidAcronym)
                     throws IOException

  Deprecated. Remove in 3.X and make true the only valid value

  Parameters:
    `stopwords` - The stopwords to use
    `replaceInvalidAcronym` - Set to true if this analyzer should replace mischaracterized acronyms in the StandardTokenizer See https://issues.apache.org/jira/browse/LUCENE-1068
  Throws:
    `IOException`

Method Detail

### getDefaultReplaceInvalidAcronym

    public static boolean getDefaultReplaceInvalidAcronym()

  Deprecated. This will be removed (hardwired to true) in 3.0

  Returns:
    true if new instances of StandardTokenizer will replace mischaracterized acronyms See https://issues.apache.org/jira/browse/LUCENE-1068

### setDefaultReplaceInvalidAcronym

    public static void setDefaultReplaceInvalidAcronym(boolean replaceInvalidAcronym)

  Deprecated. This will be removed (hardwired to true) in 3.0

  Parameters:
    `replaceInvalidAcronym` - Set to true to have new instances of StandardTokenizer replace mischaracterized acronyms by default. Set to false to preseve the previous (before 2.4) buggy behavior. Alternatively, set the system property org.apache.lucene.analysis.standard.StandardAnalyzer.replaceInvalidAcronym to false. See https://issues.apache.org/jira/browse/LUCENE-1068

### tokenStream

    public TokenStream tokenStream(String fieldName,
                                   Reader reader)

  Constructs a `StandardTokenizer` filtered by a `StandardFilter`, a `LowerCaseFilter` and a `StopFilter`.

  Specified by:
    `tokenStream` in class `Analyzer`

### setMaxTokenLength

    public void setMaxTokenLength(int length)

  Set maximum allowed token length. If a token is seen that exceeds this length then it is discarded. This setting only takes effect the next time tokenStream or reusableTokenStream is called.

### getMaxTokenLength

    public int getMaxTokenLength()

  See Also:
    `setMaxTokenLength(int)`

### reusableTokenStream

    public TokenStream reusableTokenStream(String fieldName,
                                           Reader reader)
                                    throws IOException

  Description copied from class: `Analyzer`
  Creates a TokenStream that is allowed to be re-used from the previous time that the same thread called this method. Callers that do not need to use more than one TokenStream at the same time from this analyzer should use this method for better performance.

  Overrides:
    `reusableTokenStream` in class `Analyzer`
  Throws:
    `IOException`

### isReplaceInvalidAcronym

    public boolean isReplaceInvalidAcronym()

  Deprecated. This will be removed (hardwired to true) in 3.0

  Returns:
    true if this Analyzer is replacing mischaracterized acronyms in the StandardTokenizer See https://issues.apache.org/jira/browse/LUCENE-1068

### setReplaceInvalidAcronym

    public void setReplaceInvalidAcronym(boolean replaceInvalidAcronym)

  Deprecated. This will be removed (hardwired to true) in 3.0

  Parameters:
    `replaceInvalidAcronym` - Set to true if this Analyzer is replacing mischaracterized acronyms in the StandardTokenizer See https://issues.apache.org/jira/browse/LUCENE-1068

Overview | Package | Class | Use | Tree | Deprecated | Index | Help
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

Copyright © 2000-2009 Apache Software Foundation. All Rights Reserved.