gov.sandia.cognition.learning.algorithm.perceptron
Class OnlinePassiveAggressivePerceptron.QuadraticSoftMargin

java.lang.Object
  extended by gov.sandia.cognition.util.AbstractCloneableSerializable
      extended by gov.sandia.cognition.learning.algorithm.AbstractBatchAndIncrementalLearner<InputOutputPair<? extends InputType,OutputType>,ResultType>
          extended by gov.sandia.cognition.learning.algorithm.AbstractSupervisedBatchAndIncrementalLearner<Vectorizable,Boolean,LinearBinaryCategorizer>
              extended by gov.sandia.cognition.learning.algorithm.perceptron.AbstractOnlineLinearBinaryCategorizerLearner
                  extended by gov.sandia.cognition.learning.algorithm.perceptron.AbstractKernelizableBinaryCategorizerOnlineLearner
                      extended by gov.sandia.cognition.learning.algorithm.perceptron.AbstractLinearCombinationOnlineLearner
                          extended by gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron
                              extended by gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron.AbstractSoftMargin
                                  extended by gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron.QuadraticSoftMargin
All Implemented Interfaces:
BatchAndIncrementalLearner<InputOutputPair<? extends Vectorizable,Boolean>,LinearBinaryCategorizer>, BatchLearner<Collection<? extends InputOutputPair<? extends Vectorizable,Boolean>>,LinearBinaryCategorizer>, IncrementalLearner<InputOutputPair<? extends Vectorizable,Boolean>,LinearBinaryCategorizer>, KernelizableBinaryCategorizerOnlineLearner, SupervisedBatchAndIncrementalLearner<Vectorizable,Boolean,LinearBinaryCategorizer>, SupervisedBatchLearner<Vectorizable,Boolean,LinearBinaryCategorizer>, SupervisedIncrementalLearner<Vectorizable,Boolean,LinearBinaryCategorizer>, VectorFactoryContainer, CloneableSerializable, Serializable, Cloneable
Direct Known Subclasses:
OnlineRampPassiveAggressivePerceptron
Enclosing class:
OnlinePassiveAggressivePerceptron

public static class OnlinePassiveAggressivePerceptron.QuadraticSoftMargin
extends OnlinePassiveAggressivePerceptron.AbstractSoftMargin

An implementation of the quadratic soft-margin variant of the Passive- Aggressive algorithm (PA-II). It trades-off quickly responding to errors with keeping around history in the case of non-linearly separable data.

Solves the optimization problem:
w_{t+1} = argmin_{w \in R^n} 0.5 ||w - w_t||^2 + C * \epsilon^2 such that l(w; (x_t, y_t)) <= \epslion where l(w; (x_t, y_t)) is the hinge loss (0 if y(w * x) >= 1) and 1 - y(w * x) otherwise.

The actual update step of the PA algorithm is simple: w_{t+1} = w_t + \tau_t * y_t * x_t where \tau_t = l_t / (||x_t||^2 + 0.5 * C)

Note: This algorithm does not modify the bias of the linear categorizer. Thus, it is recommended that a bias term be added to the inputs prior to use.

See Also:
Serialized Form

Nested Class Summary
 
Nested classes/interfaces inherited from class gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron
OnlinePassiveAggressivePerceptron.AbstractSoftMargin, OnlinePassiveAggressivePerceptron.LinearSoftMargin, OnlinePassiveAggressivePerceptron.QuadraticSoftMargin
 
Field Summary
 
Fields inherited from class gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron.AbstractSoftMargin
aggressiveness, DEFAULT_AGGRESSIVENESS
 
Fields inherited from class gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron
DEFAULT_UPDATE_BIAS
 
Fields inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractLinearCombinationOnlineLearner
updateBias
 
Fields inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractOnlineLinearBinaryCategorizerLearner
vectorFactory
 
Constructor Summary
OnlinePassiveAggressivePerceptron.QuadraticSoftMargin()
          Creates a new QuadraticSoftMargin with default parameters.
OnlinePassiveAggressivePerceptron.QuadraticSoftMargin(double aggressiveness)
          Creates a new QuadraticSoftMargin with the given aggressiveness.
OnlinePassiveAggressivePerceptron.QuadraticSoftMargin(double aggressiveness, VectorFactory<?> vectorFactory)
          Creates a new QuadraticSoftMargin with the given parameters.
 
Method Summary
protected  double computeUpdate(double actual, double predicted, double loss, double inputNorm2Squared)
          Compute the update value (tau) for the algorithm.
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron.AbstractSoftMargin
getAggressiveness, setAggressiveness
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron
computeUpdate, computeUpdate
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractLinearCombinationOnlineLearner
computeDecay, computeDecay, computeRescaling, computeRescaling, createInitialLearnedObject, initialize, initialize, isUpdateBias, setUpdateBias, update, update
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractKernelizableBinaryCategorizerOnlineLearner
createKernelLearner, learn, update, update, update
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractOnlineLinearBinaryCategorizerLearner
createInitialLearnedObject, getVectorFactory, setVectorFactory, update
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractSupervisedBatchAndIncrementalLearner
update
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractBatchAndIncrementalLearner
clone, learn, learn, update
 
Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface gov.sandia.cognition.learning.algorithm.SupervisedIncrementalLearner
update
 
Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchAndIncrementalLearner
learn
 
Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchLearner
learn
 
Methods inherited from interface gov.sandia.cognition.learning.algorithm.IncrementalLearner
createInitialLearnedObject, update, update
 
Methods inherited from interface gov.sandia.cognition.util.CloneableSerializable
clone
 

Constructor Detail

OnlinePassiveAggressivePerceptron.QuadraticSoftMargin

public OnlinePassiveAggressivePerceptron.QuadraticSoftMargin()
Creates a new QuadraticSoftMargin with default parameters.


OnlinePassiveAggressivePerceptron.QuadraticSoftMargin

public OnlinePassiveAggressivePerceptron.QuadraticSoftMargin(double aggressiveness)
Creates a new QuadraticSoftMargin with the given aggressiveness.

Parameters:
aggressiveness - The aggressiveness. Must be positive.

OnlinePassiveAggressivePerceptron.QuadraticSoftMargin

public OnlinePassiveAggressivePerceptron.QuadraticSoftMargin(double aggressiveness,
                                                             VectorFactory<?> vectorFactory)
Creates a new QuadraticSoftMargin with the given parameters.

Parameters:
aggressiveness - The aggressiveness. Must be positive.
vectorFactory - The factory to use to create new weight vectors.
Method Detail

computeUpdate

protected double computeUpdate(double actual,
                               double predicted,
                               double loss,
                               double inputNorm2Squared)
Description copied from class: OnlinePassiveAggressivePerceptron
Compute the update value (tau) for the algorithm. Other variants of the algorithm should override this method.

Overrides:
computeUpdate in class OnlinePassiveAggressivePerceptron
Parameters:
actual - The actual label represented as a double (-1 or +1).
predicted - The value predicted by the current categorizer (w * x + b).
loss - The loss function (1 - predicted).
inputNorm2Squared - The squared 2-norm of the input (||x||^2).
Returns:
The update value to scale the input by. Should be > 0.0. May be 0.0 in degenerate cases such as a zero-vector input.