gov.sandia.cognition.learning.algorithm.perceptron
Class BatchMultiPerceptron<CategoryType>

java.lang.Object
  extended by gov.sandia.cognition.util.AbstractCloneableSerializable
      extended by gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm
          extended by gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm<ResultType>
              extended by gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner<Collection<? extends InputOutputPair<? extends InputType,OutputType>>,ResultType>
                  extended by gov.sandia.cognition.learning.algorithm.AbstractAnytimeSupervisedBatchLearner<Vectorizable,CategoryType,LinearMultiCategorizer<CategoryType>>
                      extended by gov.sandia.cognition.learning.algorithm.perceptron.BatchMultiPerceptron<CategoryType>
Type Parameters:
CategoryType - The type of output categories. Can be any type that has a valid equals and hashCode method.
All Implemented Interfaces:
AnytimeAlgorithm<LinearMultiCategorizer<CategoryType>>, IterativeAlgorithm, MeasurablePerformanceAlgorithm, StoppableAlgorithm, AnytimeBatchLearner<Collection<? extends InputOutputPair<? extends Vectorizable,CategoryType>>,LinearMultiCategorizer<CategoryType>>, BatchLearner<Collection<? extends InputOutputPair<? extends Vectorizable,CategoryType>>,LinearMultiCategorizer<CategoryType>>, SupervisedBatchLearner<Vectorizable,CategoryType,LinearMultiCategorizer<CategoryType>>, VectorFactoryContainer, CloneableSerializable, Serializable, Cloneable

@PublicationReference(title="Ultraconservative Online Algorithms for Multiclass Problems",
                      author={"Koby Crammer","Yoram Singer"},
                      year=2003,
                      type=Journal,
                      publication="Journal of Machine Learning Research",
                      pages={951,991},
                      url="http://portal.acm.org/citation.cfm?id=944936")
public class BatchMultiPerceptron<CategoryType>
extends AbstractAnytimeSupervisedBatchLearner<Vectorizable,CategoryType,LinearMultiCategorizer<CategoryType>>
implements MeasurablePerformanceAlgorithm, VectorFactoryContainer

Implements a multi-class version of the standard batch Perceptron learning algorithm. It learns over labeled examples that have a vector input and an arbitrary output label. This version keeps a separate Perceptron as the prototype for each category. When it makes an error, it subtracts the input from the incorrect prototype and adds it to the correct prototype. This implementation also allows a margin to be enforced, which in the multi-class case means that the output value for the actual class must be at least the given margin from the next highest class.

Since:
3.2.0
Author:
Justin Basilico
See Also:
Perceptron, OnlinePerceptron, Serialized Form

Field Summary
static int DEFAULT_MAX_ITERATIONS
          The default maximum number of iterations, 100.
static double DEFAULT_MIN_MARGIN
          The default minimum margin is 0.0.
protected  int errorCount
          The number of errors on the most recent iteration.
protected  double minMargin
          The minimum margin to enforce.
protected  LinearMultiCategorizer<CategoryType> result
          The linear categorizer created by the algorithm.
protected  VectorFactory<?> vectorFactory
          The factory to create weight vectors.
 
Fields inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner
data, keepGoing
 
Fields inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm
maxIterations
 
Fields inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm
DEFAULT_ITERATION, iteration
 
Constructor Summary
BatchMultiPerceptron()
          Creates a new BatchMultiPerceptron with default parameters.
BatchMultiPerceptron(int maxIterations)
          Creates a new BatchMultiPerceptron with the given maximum number of iterations and a default margin of 0.0.
BatchMultiPerceptron(int maxIterations, double minMargin)
          Creates a new BatchMultiPerceptron with the given maximum number of iterations and margin to enforce.
BatchMultiPerceptron(int maxIterations, double minMargin, VectorFactory<?> vectorFactory)
          Creates a new BatchMultiPerceptron with the given parameters.
 
Method Summary
protected  void cleanupAlgorithm()
          Called to clean up the learning algorithm's state after learning has finished.
 int getErrorCount()
          Gets the error count of the most recent iteration.
 double getMinMargin()
          Gets the minimum margin to enforce.
 NamedValue<Integer> getPerformance()
          Gets the performance, which is the error count on the last iteration.
 LinearMultiCategorizer<CategoryType> getResult()
          Gets the current result of the algorithm.
 VectorFactory<?> getVectorFactory()
          Gets the VectorFactory used to create the weight vector.
protected  boolean initializeAlgorithm()
          Called to initialize the learning algorithm's state based on the data that is stored in the data field.
protected  void setErrorCount(int errorCount)
          Sets the error count of the most recent iteration.
 void setMinMargin(double minMargin)
          Gets the minimum margin to enforce.
protected  void setResult(LinearMultiCategorizer<CategoryType> result)
          Sets the result of the algorithm.
 void setVectorFactory(VectorFactory<?> vectorFactory)
          Sets the VectorFactory used to create the weight vector.
protected  boolean step()
          Called to take a single step of the learning algorithm.
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner
clone, getData, getKeepGoing, learn, setData, setKeepGoing, stop
 
Methods inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm
getMaxIterations, isResultValid, setMaxIterations
 
Methods inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm
addIterativeAlgorithmListener, fireAlgorithmEnded, fireAlgorithmStarted, fireStepEnded, fireStepStarted, getIteration, getListeners, removeIterativeAlgorithmListener, setIteration, setListeners
 
Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchLearner
learn
 
Methods inherited from interface gov.sandia.cognition.util.CloneableSerializable
clone
 
Methods inherited from interface gov.sandia.cognition.algorithm.AnytimeAlgorithm
getMaxIterations, setMaxIterations
 
Methods inherited from interface gov.sandia.cognition.algorithm.IterativeAlgorithm
addIterativeAlgorithmListener, getIteration, removeIterativeAlgorithmListener
 
Methods inherited from interface gov.sandia.cognition.algorithm.StoppableAlgorithm
isResultValid
 

Field Detail

DEFAULT_MAX_ITERATIONS

public static final int DEFAULT_MAX_ITERATIONS
The default maximum number of iterations, 100.

See Also:
Constant Field Values

DEFAULT_MIN_MARGIN

public static final double DEFAULT_MIN_MARGIN
The default minimum margin is 0.0.

See Also:
Constant Field Values

minMargin

protected double minMargin
The minimum margin to enforce. Must be non-negative.


vectorFactory

protected VectorFactory<?> vectorFactory
The factory to create weight vectors.


result

protected transient LinearMultiCategorizer<CategoryType> result
The linear categorizer created by the algorithm.


errorCount

protected transient int errorCount
The number of errors on the most recent iteration.

Constructor Detail

BatchMultiPerceptron

public BatchMultiPerceptron()
Creates a new BatchMultiPerceptron with default parameters.


BatchMultiPerceptron

public BatchMultiPerceptron(int maxIterations)
Creates a new BatchMultiPerceptron with the given maximum number of iterations and a default margin of 0.0.

Parameters:
maxIterations - The maximum number of iterations. Must be positive.

BatchMultiPerceptron

public BatchMultiPerceptron(int maxIterations,
                            double minMargin)
Creates a new BatchMultiPerceptron with the given maximum number of iterations and margin to enforce.

Parameters:
maxIterations - The maximum number of iterations. Must be positive.
minMargin - The minimum margin to enforce. Must be non-negative.

BatchMultiPerceptron

public BatchMultiPerceptron(int maxIterations,
                            double minMargin,
                            VectorFactory<?> vectorFactory)
Creates a new BatchMultiPerceptron with the given parameters.

Parameters:
maxIterations - The maximum number of iterations. Must be positive.
minMargin - The minimum margin to enforce. Must be non-negative.
vectorFactory - The factory to use to create weight vectors.
Method Detail

initializeAlgorithm

protected boolean initializeAlgorithm()
Description copied from class: AbstractAnytimeBatchLearner
Called to initialize the learning algorithm's state based on the data that is stored in the data field. The return value indicates if the algorithm can be run or not based on the initialization.

Specified by:
initializeAlgorithm in class AbstractAnytimeBatchLearner<Collection<? extends InputOutputPair<? extends Vectorizable,CategoryType>>,LinearMultiCategorizer<CategoryType>>
Returns:
True if the learning algorithm can be run and false if it cannot.

step

protected boolean step()
Description copied from class: AbstractAnytimeBatchLearner
Called to take a single step of the learning algorithm.

Specified by:
step in class AbstractAnytimeBatchLearner<Collection<? extends InputOutputPair<? extends Vectorizable,CategoryType>>,LinearMultiCategorizer<CategoryType>>
Returns:
True if another step can be taken and false it the algorithm should halt.

cleanupAlgorithm

protected void cleanupAlgorithm()
Description copied from class: AbstractAnytimeBatchLearner
Called to clean up the learning algorithm's state after learning has finished.

Specified by:
cleanupAlgorithm in class AbstractAnytimeBatchLearner<Collection<? extends InputOutputPair<? extends Vectorizable,CategoryType>>,LinearMultiCategorizer<CategoryType>>

getResult

public LinearMultiCategorizer<CategoryType> getResult()
Description copied from interface: AnytimeAlgorithm
Gets the current result of the algorithm.

Specified by:
getResult in interface AnytimeAlgorithm<LinearMultiCategorizer<CategoryType>>
Returns:
Current result of the algorithm.

setResult

protected void setResult(LinearMultiCategorizer<CategoryType> result)
Sets the result of the algorithm.

Parameters:
result - The result of the algorithm.

getMinMargin

public double getMinMargin()
Gets the minimum margin to enforce. Any value less than or equal to this is considered an error for the algorithm.

Returns:
The minimum margin. Cannot be negative.

setMinMargin

public void setMinMargin(double minMargin)
Gets the minimum margin to enforce. Any value less than or equal to this is considered an error for the algorithm.

Parameters:
minMargin - The minimum margin. Cannot be negative.

getVectorFactory

public VectorFactory<?> getVectorFactory()
Gets the VectorFactory used to create the weight vector.

Specified by:
getVectorFactory in interface VectorFactoryContainer
Returns:
The VectorFactory used to create the weight vector.

setVectorFactory

public void setVectorFactory(VectorFactory<?> vectorFactory)
Sets the VectorFactory used to create the weight vector.

Parameters:
vectorFactory - The VectorFactory used to create the weight vector.

getErrorCount

public int getErrorCount()
Gets the error count of the most recent iteration.

Returns:
The current error count.

setErrorCount

protected void setErrorCount(int errorCount)
Sets the error count of the most recent iteration.

Parameters:
errorCount - The current error count.

getPerformance

public NamedValue<Integer> getPerformance()
Gets the performance, which is the error count on the last iteration.

Specified by:
getPerformance in interface MeasurablePerformanceAlgorithm
Returns:
The performance of the algorithm.