gov.sandia.cognition.learning.algorithm.perceptron
Class OnlinePassiveAggressivePerceptron.QuadraticSoftMargin
java.lang.Object
gov.sandia.cognition.util.AbstractCloneableSerializable
gov.sandia.cognition.learning.algorithm.AbstractBatchAndIncrementalLearner<InputOutputPair<? extends InputType,OutputType>,ResultType>
gov.sandia.cognition.learning.algorithm.AbstractSupervisedBatchAndIncrementalLearner<Vectorizable,Boolean,LinearBinaryCategorizer>
gov.sandia.cognition.learning.algorithm.perceptron.AbstractOnlineLinearBinaryCategorizerLearner
gov.sandia.cognition.learning.algorithm.perceptron.AbstractKernelizableBinaryCategorizerOnlineLearner
gov.sandia.cognition.learning.algorithm.perceptron.AbstractLinearCombinationOnlineLearner
gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron
gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron.AbstractSoftMargin
gov.sandia.cognition.learning.algorithm.perceptron.OnlinePassiveAggressivePerceptron.QuadraticSoftMargin
- All Implemented Interfaces:
- BatchAndIncrementalLearner<InputOutputPair<? extends Vectorizable,Boolean>,LinearBinaryCategorizer>, BatchLearner<Collection<? extends InputOutputPair<? extends Vectorizable,Boolean>>,LinearBinaryCategorizer>, IncrementalLearner<InputOutputPair<? extends Vectorizable,Boolean>,LinearBinaryCategorizer>, KernelizableBinaryCategorizerOnlineLearner, SupervisedBatchAndIncrementalLearner<Vectorizable,Boolean,LinearBinaryCategorizer>, SupervisedBatchLearner<Vectorizable,Boolean,LinearBinaryCategorizer>, SupervisedIncrementalLearner<Vectorizable,Boolean,LinearBinaryCategorizer>, VectorFactoryContainer, CloneableSerializable, Serializable, Cloneable
- Direct Known Subclasses:
- OnlineRampPassiveAggressivePerceptron
- Enclosing class:
- OnlinePassiveAggressivePerceptron
public static class OnlinePassiveAggressivePerceptron.QuadraticSoftMargin
- extends OnlinePassiveAggressivePerceptron.AbstractSoftMargin
An implementation of the quadratic soft-margin variant of the Passive-
Aggressive algorithm (PA-II). It trades-off quickly responding to errors
with keeping around history in the case of non-linearly separable data.
Solves the optimization problem:
w_{t+1} = argmin_{w \in R^n} 0.5 ||w - w_t||^2 + C * \epsilon^2
such that
l(w; (x_t, y_t)) <= \epslion
where l(w; (x_t, y_t)) is the hinge loss (0 if y(w * x) >= 1) and
1 - y(w * x) otherwise.
The actual update step of the PA algorithm is simple:
w_{t+1} = w_t + \tau_t * y_t * x_t
where
\tau_t = l_t / (||x_t||^2 + 0.5 * C)
Note: This algorithm does not modify the bias of the linear categorizer.
Thus, it is recommended that a bias term be added to the inputs prior to
use.
- See Also:
- Serialized Form
|
Method Summary |
protected double |
computeUpdate(double actual,
double predicted,
double loss,
double inputNorm2Squared)
Compute the update value (tau) for the algorithm. |
| Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractLinearCombinationOnlineLearner |
computeDecay, computeDecay, computeRescaling, computeRescaling, createInitialLearnedObject, initialize, initialize, isUpdateBias, setUpdateBias, update, update |
| Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchLearner |
learn |
OnlinePassiveAggressivePerceptron.QuadraticSoftMargin
public OnlinePassiveAggressivePerceptron.QuadraticSoftMargin()
- Creates a new
QuadraticSoftMargin with default parameters.
OnlinePassiveAggressivePerceptron.QuadraticSoftMargin
public OnlinePassiveAggressivePerceptron.QuadraticSoftMargin(double aggressiveness)
- Creates a new
QuadraticSoftMargin with the given
aggressiveness.
- Parameters:
aggressiveness - The aggressiveness. Must be positive.
OnlinePassiveAggressivePerceptron.QuadraticSoftMargin
public OnlinePassiveAggressivePerceptron.QuadraticSoftMargin(double aggressiveness,
VectorFactory<?> vectorFactory)
- Creates a new
QuadraticSoftMargin with the given parameters.
- Parameters:
aggressiveness - The aggressiveness. Must be positive.vectorFactory - The factory to use to create new weight vectors.
computeUpdate
protected double computeUpdate(double actual,
double predicted,
double loss,
double inputNorm2Squared)
- Description copied from class:
OnlinePassiveAggressivePerceptron
- Compute the update value (tau) for the algorithm. Other variants of
the algorithm should override this method.
- Overrides:
computeUpdate in class OnlinePassiveAggressivePerceptron
- Parameters:
actual - The actual label represented as a double (-1 or +1).predicted - The value predicted by the current categorizer (w * x + b).loss - The loss function (1 - predicted).inputNorm2Squared - The squared 2-norm of the input (||x||^2).
- Returns:
- The update value to scale the input by. Should be > 0.0. May be 0.0
in degenerate cases such as a zero-vector input.