gov.sandia.cognition.learning.algorithm.pca
Class GeneralizedHebbianAlgorithm

java.lang.Object
  extended by gov.sandia.cognition.util.AbstractCloneableSerializable
      extended by gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm
          extended by gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm<ResultType>
              extended by gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner<Collection<Vector>,PrincipalComponentsAnalysisFunction>
                  extended by gov.sandia.cognition.learning.algorithm.pca.GeneralizedHebbianAlgorithm
All Implemented Interfaces:
AnytimeAlgorithm<PrincipalComponentsAnalysisFunction>, IterativeAlgorithm, MeasurablePerformanceAlgorithm, StoppableAlgorithm, AnytimeBatchLearner<Collection<Vector>,PrincipalComponentsAnalysisFunction>, BatchLearner<Collection<Vector>,PrincipalComponentsAnalysisFunction>, PrincipalComponentsAnalysis, CloneableSerializable, Serializable, Cloneable

@CodeReview(reviewer="Kevin R. Dixon",
            date="2008-07-23",
            changesNeeded=false,
            comments={"Added PublicationReference to Sanger\'s master\'s thesis.","Minor changes to javadoc.","Looks fine."})
@PublicationReference(author="Terrence D. Sanger",
                      title="Optimal Unsupervised Learning in a Single-Layer Linear Feedforward Neural Network",
                      type=Thesis,
                      year=1989,
                      url="http://ece-classweb.ucsd.edu/winter06/ece173/documents/Sanger%201989%20--%20Optimal%20Unsupervised%20Learning%20in%20a%20Single-layer%20Linear%20FeedforwardNN.pdf")
public class GeneralizedHebbianAlgorithm
extends AbstractAnytimeBatchLearner<Collection<Vector>,PrincipalComponentsAnalysisFunction>
implements PrincipalComponentsAnalysis, MeasurablePerformanceAlgorithm

Implementation of the Generalized Hebbian Algorithm, also known as Sanger's Rule, which is a generalization of Oja's Rule. This algorithm is an iterative version of Principal Component Analysis. GHA finds the "num" Vectors corresponding to the "num" largest singular values of the covariance matrix of the data. The result is a VectorFunction that maps the input space onto a reduced "num" dimensional space, which captures the directions of maximal variance. The ith row in the resulting matrix approximates the i-th column of the "U" matrix of the Singular Value Decomposition. Amazingly, this implementation is faster than the time taken to perform closed-form SVD on datasets, and is practical on datasets too large for an SVD.

Since:
2.0
Author:
Kevin R. Dixon
See Also:
Serialized Form

Field Summary
static String PERFORMANCE_NAME
          The performance name is "Change".
 
Fields inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner
data, keepGoing
 
Fields inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm
maxIterations
 
Fields inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm
DEFAULT_ITERATION, iteration
 
Constructor Summary
GeneralizedHebbianAlgorithm(int numComponents, double learningRate, int maxIterations, double minChange)
          Creates a new instance of GeneralizedHebbianAlgorithm
 
Method Summary
protected  void cleanupAlgorithm()
          Called to clean up the learning algorithm's state after learning has finished.
 GeneralizedHebbianAlgorithm clone()
          This makes public the clone method on the Object class and removes the exception that it throws.
 double getChange()
          Gets the change in in the last completed step of the algorithm.
 double getLearningRate()
          Getter for learningRate
 double getMinChange()
          Getter for minChange
 int getNumComponents()
          Gets the number of components used in the PCA dimension reduction.
 NamedValue<Double> getPerformance()
          Gets the name-value pair that describes the current performance of the algorithm.
 PrincipalComponentsAnalysisFunction getResult()
          Gets the VectorFunction that maps from the input space to the reduced output space of "getNumComponents" dimensions.
protected  boolean initializeAlgorithm()
          Called to initialize the learning algorithm's state based on the data that is stored in the data field.
 void setLearningRate(double learningRate)
          Setter for learningRate
 void setMinChange(double minChange)
          Setter for minChange
 void setNumComponents(int numComponents)
          Setter for numComponents
protected  void setResult(PrincipalComponentsAnalysisFunction result)
          Setter for result
protected  boolean step()
          Called to take a single step of the learning algorithm.
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner
getData, getKeepGoing, learn, setData, setKeepGoing, stop
 
Methods inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm
getMaxIterations, isResultValid, setMaxIterations
 
Methods inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm
addIterativeAlgorithmListener, fireAlgorithmEnded, fireAlgorithmStarted, fireStepEnded, fireStepStarted, getIteration, getListeners, removeIterativeAlgorithmListener, setIteration, setListeners
 
Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchLearner
learn
 
Methods inherited from interface gov.sandia.cognition.algorithm.AnytimeAlgorithm
getMaxIterations, setMaxIterations
 
Methods inherited from interface gov.sandia.cognition.algorithm.IterativeAlgorithm
addIterativeAlgorithmListener, getIteration, removeIterativeAlgorithmListener
 
Methods inherited from interface gov.sandia.cognition.algorithm.StoppableAlgorithm
isResultValid
 

Field Detail

PERFORMANCE_NAME

public static final String PERFORMANCE_NAME
The performance name is "Change".

See Also:
Constant Field Values
Constructor Detail

GeneralizedHebbianAlgorithm

public GeneralizedHebbianAlgorithm(int numComponents,
                                   double learningRate,
                                   int maxIterations,
                                   double minChange)
Creates a new instance of GeneralizedHebbianAlgorithm

Parameters:
minChange - Minimum change below which to stop iterating, greater than or equal to zero, typically 1e-10
numComponents - Number of components to extract from the data, must be greater than zero
learningRate - Learning rate, or step size, (0,1], typically ~0.1
maxIterations - Maximum number of iterations before stopping
Method Detail

clone

public GeneralizedHebbianAlgorithm clone()
Description copied from class: AbstractCloneableSerializable
This makes public the clone method on the Object class and removes the exception that it throws. Its default behavior is to automatically create a clone of the exact type of object that the clone is called on and to copy all primitives but to keep all references, which means it is a shallow copy. Extensions of this class may want to override this method (but call super.clone() to implement a "smart copy". That is, to target the most common use case for creating a copy of the object. Because of the default behavior being a shallow copy, extending classes only need to handle fields that need to have a deeper copy (or those that need to be reset). Some of the methods in ObjectUtil may be helpful in implementing a custom clone method. Note: The contract of this method is that you must use super.clone() as the basis for your implementation.

Specified by:
clone in interface CloneableSerializable
Overrides:
clone in class AbstractAnytimeBatchLearner<Collection<Vector>,PrincipalComponentsAnalysisFunction>
Returns:
A clone of this object.

initializeAlgorithm

protected boolean initializeAlgorithm()
Description copied from class: AbstractAnytimeBatchLearner
Called to initialize the learning algorithm's state based on the data that is stored in the data field. The return value indicates if the algorithm can be run or not based on the initialization.

Specified by:
initializeAlgorithm in class AbstractAnytimeBatchLearner<Collection<Vector>,PrincipalComponentsAnalysisFunction>
Returns:
True if the learning algorithm can be run and false if it cannot.

cleanupAlgorithm

protected void cleanupAlgorithm()
Description copied from class: AbstractAnytimeBatchLearner
Called to clean up the learning algorithm's state after learning has finished.

Specified by:
cleanupAlgorithm in class AbstractAnytimeBatchLearner<Collection<Vector>,PrincipalComponentsAnalysisFunction>

step

protected boolean step()
Description copied from class: AbstractAnytimeBatchLearner
Called to take a single step of the learning algorithm.

Specified by:
step in class AbstractAnytimeBatchLearner<Collection<Vector>,PrincipalComponentsAnalysisFunction>
Returns:
True if another step can be taken and false it the algorithm should halt.

getLearningRate

public double getLearningRate()
Getter for learningRate

Returns:
Learning rate, or step size, (0,1], typically ~0.1

setLearningRate

public void setLearningRate(double learningRate)
Setter for learningRate

Parameters:
learningRate - Learning rate, or step size, (0,1], typically ~0.1

getMinChange

public double getMinChange()
Getter for minChange

Returns:
Minimum change below which to stop iterating, greater than or equal to zero, typically 1e-10

setMinChange

public void setMinChange(double minChange)
Setter for minChange

Parameters:
minChange - Minimum change below which to stop iterating, greater than or equal to zero, typically 1e-10

getNumComponents

public int getNumComponents()
Description copied from interface: PrincipalComponentsAnalysis
Gets the number of components used in the PCA dimension reduction.

Specified by:
getNumComponents in interface PrincipalComponentsAnalysis
Returns:
Number of components used in the PCA dimension reduction

setNumComponents

public void setNumComponents(int numComponents)
Setter for numComponents

Parameters:
numComponents - Number of components to extract from the data, must be greater than zero

getResult

public PrincipalComponentsAnalysisFunction getResult()
Description copied from interface: PrincipalComponentsAnalysis
Gets the VectorFunction that maps from the input space to the reduced output space of "getNumComponents" dimensions.

Specified by:
getResult in interface AnytimeAlgorithm<PrincipalComponentsAnalysisFunction>
Specified by:
getResult in interface PrincipalComponentsAnalysis
Returns:
PCA function that reduces the dimensionality of the input space to a (hopefully) simpler and smaller output space

setResult

protected void setResult(PrincipalComponentsAnalysisFunction result)
Setter for result

Parameters:
result - Vector function that maps the input space onto a numComponents-dimension Vector representing the directions of maximal variance (information gain). The ith row in the matrix approximates the i-th column of the "U" matrix of the Singular Value Decomposition.

getChange

public double getChange()
Gets the change in in the last completed step of the algorithm.

Returns:
The change in the last completed step of the algorithm.

getPerformance

public NamedValue<Double> getPerformance()
Description copied from interface: MeasurablePerformanceAlgorithm
Gets the name-value pair that describes the current performance of the algorithm. For most algorithms, this is the value that they are attempting to optimize.

Specified by:
getPerformance in interface MeasurablePerformanceAlgorithm
Returns:
The name-value pair that describes the current performance of the algorithm.