emolib.stemmer
Class Stemmer

java.lang.Object
  emolib.util.proc.TextDataProcessor
      emolib.stemmer.Stemmer

All Implemented Interfaces:

Configurable, DataProcessor

Direct Known Subclasses:

GenericSnowballStemmer

public abstract class Stemmer
extends TextDataProcessor

The Stemmer abstract class defines the general structure to perform the stemming process, which eliminates the inflexion of words.

The idea of stemming is to improve the Information Retrieval (IR) performance generally by bringing under one heading variant forms of a word which share a common meaning.

Stemming is feasible to the Indo-European languages because there exist common patterns of word structures, and for languages that are more highly inflected than English (and most of them are), greater improvements will be observed when stemming is applied. Thus, for a Romance language like Spanish the results are expected to be reasonably good.

Assuming that words are written left to right, the stem, or root of a word is on the left, and zero or more suffixes may be added on the right. If the root is modified by this process it will normally be at its right hand end. Also, prefixes may be added on the left usually altering its meaning radically, so they are best left in place. But suffixes can, in certain circumstances, be removed. In fact, suffix stripping is a practical aid in IR after all.

Here, stem and root are used interchangeably, but there exists a finer distinction between them, regarding the stem as the residue of the stemming process and the root as the inner word from which the stem word derives. Anyway, such a proficient degree is not intended in this Stemmer class description.

There is a lot more to be said about the stemming process: the stemming errors, the use of dictionaries, stop words, irregularities, etc. Refer to Dr. Porter's article Snowball: A language for stemming algorithms in order to get an extensive description of the stemming process.

Author:

Alexandre Trilla (atrilla@salle.url.edu)

Constructor Summary

Stemmer()
Main constructor of the Stemmer.

Method Summary

abstract void	applyStemming(TextData inputTextDataObject) Method to perform the stemming process.
Data	getData() Obtains the TextData from the previous module, processes it and makes it available to the rest of the text processing chain.
void	initialize() Method to initialize the Stemmer.
void	newProperties(PropertySheet ps) This method is called when this configurable component has new data.
void	register(java.lang.String name, Registry registry) Register my properties.

Methods inherited from class emolib.util.proc.TextDataProcessor

flush, getName, getPredecessor, setPredecessor, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

Stemmer

public Stemmer()

Main constructor of the Stemmer.

Method Detail

register

public void register(java.lang.String name,
                     Registry registry)
              throws PropertyException

Description copied from interface: Configurable

Register my properties. This method is called once early in the time of the component, shortly after the component is constructed. This component should register any configuration properties that it needs to register. If this configurable extends another configurable, super.register should also be called

Specified by:

register in interface Configurable

Overrides:

register in class TextDataProcessor

Parameters:

name - the name of the component

registry - the registry for this component

Throws:

PropertyException

newProperties

public void newProperties(PropertySheet ps)
                   throws PropertyException

Description copied from interface: Configurable

This method is called when this configurable component has new data. The component should first validate the data. If it is bad the component should return false. If the data is good, the component should record the the data internally and return true.

Specified by:

newProperties in interface Configurable

Overrides:

newProperties in class TextDataProcessor

Parameters:

ps - a property sheet holding the new data

Throws:

PropertyException - if there is a problem with the properties.

getData

public Data getData()
             throws DataProcessingException

Obtains the TextData from the previous module, processes it and makes it available to the rest of the text processing chain.

Specified by:

getData in interface DataProcessor

Specified by:

getData in class TextDataProcessor

Returns:

The next available Data object, returns null if no Data object is available.

Throws:

DataProcessingException - If there is a processing error.

initialize

public void initialize()

Method to initialize the Stemmer.

Specified by:

initialize in interface DataProcessor

Overrides:

initialize in class TextDataProcessor

applyStemming

public abstract void applyStemming(TextData inputTextDataObject)

Method to perform the stemming process.

Parameters:

inputTextDataObject - The TextData object to process.