gov.sandia.cognition.learning.algorithm.perceptron
Class OnlineShiftingPerceptron

java.lang.Object
  extended by gov.sandia.cognition.util.AbstractCloneableSerializable
      extended by gov.sandia.cognition.learning.algorithm.AbstractBatchAndIncrementalLearner<InputOutputPair<? extends InputType,OutputType>,ResultType>
          extended by gov.sandia.cognition.learning.algorithm.AbstractSupervisedBatchAndIncrementalLearner<Vectorizable,Boolean,LinearBinaryCategorizer>
              extended by gov.sandia.cognition.learning.algorithm.perceptron.AbstractOnlineLinearBinaryCategorizerLearner
                  extended by gov.sandia.cognition.learning.algorithm.perceptron.AbstractKernelizableBinaryCategorizerOnlineLearner
                      extended by gov.sandia.cognition.learning.algorithm.perceptron.AbstractLinearCombinationOnlineLearner
                          extended by gov.sandia.cognition.learning.algorithm.perceptron.OnlineShiftingPerceptron
All Implemented Interfaces:
BatchAndIncrementalLearner<InputOutputPair<? extends Vectorizable,Boolean>,LinearBinaryCategorizer>, BatchLearner<Collection<? extends InputOutputPair<? extends Vectorizable,Boolean>>,LinearBinaryCategorizer>, IncrementalLearner<InputOutputPair<? extends Vectorizable,Boolean>,LinearBinaryCategorizer>, KernelizableBinaryCategorizerOnlineLearner, SupervisedBatchAndIncrementalLearner<Vectorizable,Boolean,LinearBinaryCategorizer>, SupervisedBatchLearner<Vectorizable,Boolean,LinearBinaryCategorizer>, SupervisedIncrementalLearner<Vectorizable,Boolean,LinearBinaryCategorizer>, VectorFactoryContainer, CloneableSerializable, Serializable, Cloneable

@PublicationReference(author={"Giovanni Cavallanti","Nicolo Cesa-Bianchi","Claudio Gentile"},
                      title="Tracking the best hyperplane with a simple budget Perceptron",
                      year=2007,
                      type=Journal,
                      publication="Machine Learning",
                      pages={143,167},
                      url="http://www.springerlink.com/index/H40NV525LX161227.pdf")
public class OnlineShiftingPerceptron
extends AbstractLinearCombinationOnlineLearner

An implementation of the Shifting Perceptron algorithm. It is like the Perceptron except that it adds a decay to the existing weights, which is parameterized by lambda.

Since:
3.3.0
Author:
Justin Basilico
See Also:
Serialized Form

Nested Class Summary
static class OnlineShiftingPerceptron.LinearResult
          This is the result learned by the shifting perceptron.
 
Field Summary
static double DEFAULT_LAMBDA
          The default value of lambda is 0.001.
static boolean DEFAULT_UPDATE_BIAS
          Algorithm does not update the bias by default.
protected  double lambda
          The lambda parameter for controlling how much shifting occurs.
 
Fields inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractLinearCombinationOnlineLearner
updateBias
 
Fields inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractOnlineLinearBinaryCategorizerLearner
vectorFactory
 
Constructor Summary
OnlineShiftingPerceptron()
          Creates a new OnlineShiftingPerceptron with default parameters.
OnlineShiftingPerceptron(double lambda)
          Creates a new OnlineShiftingPerceptron with the given parameters.
OnlineShiftingPerceptron(double lambda, VectorFactory<?> vectorFactory)
          Creates a new OnlineShiftingPerceptron with the given parameters.
 
Method Summary
protected
<InputType>
double
computeDecay(DefaultKernelBinaryCategorizer<InputType> target, InputType input, boolean actualCategory, double predicted, double update)
          Computes the decay scalar for the existing weights.
protected  double computeDecay(LinearBinaryCategorizer target, Vector input, boolean actualCategory, double predicted, double update)
          Computes the decay scalar for the existing weight vector.
<InputType>
double
computeUpdate(DefaultKernelBinaryCategorizer<InputType> target, InputType input, boolean label, double predicted)
          Compute the update weight in the linear case.
 double computeUpdate(LinearBinaryCategorizer target, Vector input, boolean label, double predicted)
          Compute the update weight in the linear case.
 LinearBinaryCategorizer createInitialLearnedObject()
          Creates a new initial learned object, before any data is given.
 double getLambda()
          Gets the lambda parameter, which controls how much shifting and decay there is in the weight vector.
 void setLambda(double lambda)
          Sets the lambda parameter, which controls how much shifting and decay there is in the weight vector.
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractLinearCombinationOnlineLearner
computeRescaling, computeRescaling, createInitialLearnedObject, initialize, initialize, isUpdateBias, setUpdateBias, update, update
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractKernelizableBinaryCategorizerOnlineLearner
createKernelLearner, learn, update, update, update
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.perceptron.AbstractOnlineLinearBinaryCategorizerLearner
getVectorFactory, setVectorFactory, update
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractSupervisedBatchAndIncrementalLearner
update
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractBatchAndIncrementalLearner
clone, learn, learn, update
 
Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface gov.sandia.cognition.learning.algorithm.SupervisedIncrementalLearner
update
 
Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchAndIncrementalLearner
learn
 
Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchLearner
learn
 
Methods inherited from interface gov.sandia.cognition.learning.algorithm.IncrementalLearner
update, update
 
Methods inherited from interface gov.sandia.cognition.util.CloneableSerializable
clone
 

Field Detail

DEFAULT_UPDATE_BIAS

public static final boolean DEFAULT_UPDATE_BIAS
Algorithm does not update the bias by default.

See Also:
Constant Field Values

DEFAULT_LAMBDA

public static final double DEFAULT_LAMBDA
The default value of lambda is 0.001.

See Also:
Constant Field Values

lambda

protected double lambda
The lambda parameter for controlling how much shifting occurs.

Constructor Detail

OnlineShiftingPerceptron

public OnlineShiftingPerceptron()
Creates a new OnlineShiftingPerceptron with default parameters.


OnlineShiftingPerceptron

public OnlineShiftingPerceptron(double lambda)
Creates a new OnlineShiftingPerceptron with the given parameters.

Parameters:
lambda - The lambda parameter to control the amount of shift. Must be positive.

OnlineShiftingPerceptron

public OnlineShiftingPerceptron(double lambda,
                                VectorFactory<?> vectorFactory)
Creates a new OnlineShiftingPerceptron with the given parameters.

Parameters:
lambda - The lambda parameter to control the amount of shift. Must be positive.
vectorFactory - The vector factory to use.
Method Detail

createInitialLearnedObject

public LinearBinaryCategorizer createInitialLearnedObject()
Description copied from interface: IncrementalLearner
Creates a new initial learned object, before any data is given.

Specified by:
createInitialLearnedObject in interface IncrementalLearner<InputOutputPair<? extends Vectorizable,Boolean>,LinearBinaryCategorizer>
Overrides:
createInitialLearnedObject in class AbstractOnlineLinearBinaryCategorizerLearner
Returns:
The initial learned object.

computeUpdate

public double computeUpdate(LinearBinaryCategorizer target,
                            Vector input,
                            boolean label,
                            double predicted)
Description copied from class: AbstractLinearCombinationOnlineLearner
Compute the update weight in the linear case. Must be implemented by subclasses.

Specified by:
computeUpdate in class AbstractLinearCombinationOnlineLearner
Parameters:
target - Target to compute the update for.
input - Input to use in computing the update.
label - The actual category of the input.
predicted - The predicted category of the input.
Returns:
The update weight for how much to add the input to the target. May be zero if no update is needed.

computeUpdate

public <InputType> double computeUpdate(DefaultKernelBinaryCategorizer<InputType> target,
                                        InputType input,
                                        boolean label,
                                        double predicted)
Description copied from class: AbstractLinearCombinationOnlineLearner
Compute the update weight in the linear case. Must be implemented by subclasses.

Specified by:
computeUpdate in class AbstractLinearCombinationOnlineLearner
Type Parameters:
InputType - The input value for learning.
Parameters:
target - Target to compute the update for.
input - Input to use in computing the update.
label - The actual category of the input.
predicted - The predicted category of the input.
Returns:
The update weight for how much to add the input to the target. May be zero if no update is needed.

computeDecay

protected double computeDecay(LinearBinaryCategorizer target,
                              Vector input,
                              boolean actualCategory,
                              double predicted,
                              double update)
Description copied from class: AbstractLinearCombinationOnlineLearner
Computes the decay scalar for the existing weight vector. Can be overridden. The default implementation just returns 1.0, which means no change. Typically this will be a value between 0.0 and 1.0.

Overrides:
computeDecay in class AbstractLinearCombinationOnlineLearner
Parameters:
target - Target to compute the update for.
input - Input to use in computing the update.
actualCategory - The actual category of the input.
predicted - The predicted category of the input.
update - The value from the computeUpdate step.
Returns:
The decay to apply to the weight vector. Usually between 0.0 and 1.0.

computeDecay

protected <InputType> double computeDecay(DefaultKernelBinaryCategorizer<InputType> target,
                                          InputType input,
                                          boolean actualCategory,
                                          double predicted,
                                          double update)
Description copied from class: AbstractLinearCombinationOnlineLearner
Computes the decay scalar for the existing weights. Can be overridden. The default implementation just returns 1.0, which means no change. Typically this will be a value between 0.0 and 1.0.

Overrides:
computeDecay in class AbstractLinearCombinationOnlineLearner
Type Parameters:
InputType - The input value for learning.
Parameters:
target - Target to compute the update for.
input - Input to use in computing the update.
actualCategory - The actual category of the input.
predicted - The predicted category of the input.
update - The value from the computeUpdate step.
Returns:
The decay to apply to the weights. Usually between 0.0 and 1.0.

getLambda

public double getLambda()
Gets the lambda parameter, which controls how much shifting and decay there is in the weight vector.

Returns:
The lambda parameter. Must be positive.

setLambda

public void setLambda(double lambda)
Sets the lambda parameter, which controls how much shifting and decay there is in the weight vector.

Parameters:
lambda - The lambda parameter. Must be positive.