gov.sandia.cognition.learning.algorithm.regression
Class KernelBasedIterativeRegression<InputType>

java.lang.Object
  extended by gov.sandia.cognition.util.AbstractCloneableSerializable
      extended by gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm
          extended by gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm<ResultType>
              extended by gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner<Collection<? extends InputOutputPair<? extends InputType,OutputType>>,ResultType>
                  extended by gov.sandia.cognition.learning.algorithm.AbstractAnytimeSupervisedBatchLearner<InputType,Double,KernelScalarFunction<InputType>>
                      extended by gov.sandia.cognition.learning.algorithm.regression.KernelBasedIterativeRegression<InputType>
Type Parameters:
InputType - Input parameter to the Kernels
All Implemented Interfaces:
AnytimeAlgorithm<KernelScalarFunction<InputType>>, IterativeAlgorithm, MeasurablePerformanceAlgorithm, StoppableAlgorithm, AnytimeBatchLearner<Collection<? extends InputOutputPair<? extends InputType,Double>>,KernelScalarFunction<InputType>>, BatchLearner<Collection<? extends InputOutputPair<? extends InputType,Double>>,KernelScalarFunction<InputType>>, SupervisedBatchLearner<InputType,Double,KernelScalarFunction<InputType>>, CloneableSerializable, Serializable, Cloneable

@PublicationReference(author={"John Shawe-Taylor","Nello Cristianini"},
                      title="Kernel Methods for Pattern Analysis",
                      type=Book,
                      year=2004,
                      url="http://www.kernel-methods.net/")
public class KernelBasedIterativeRegression<InputType>
extends AbstractAnytimeSupervisedBatchLearner<InputType,Double,KernelScalarFunction<InputType>>
implements MeasurablePerformanceAlgorithm

The KernelBasedIterativeRegression class implements an online version of the Support Vector Regression algorithm. It learns a scalar kernel function that uses the given kernel to map inputs onto real numbers. The code is based on the pseudocode in the book "Kernel Methods for Pattern Analysis" by J. Shawe-Taylor and N. Cristianini. However, this the pseudo-code in the book is incorrect and seems to be missing an extra division by the kernel of the example with itself. It also does not check to make sure that the error is outside of the minimum sensitivity range. This implementation also includes a bias term, which is also omitted from the pseudo-code in the book.

The update to the weight that is implemented is:
alpha_i = alpha_i + (y_i + epsilon * sign(alpha_i) + sum alpha_j k(x_j, x_i) + b) / k(x_i, x_i)

The loss function underlying the implementation is the epsilon-insensitive loss. This parameter is named minSensitivity in this implementation. It means that errors less than or equal to minSensitivity are ignored.

Since:
2.0
Author:
Justin Basilico
See Also:
Serialized Form

Field Summary
static int DEFAULT_MAX_ITERATIONS
          The default maximum number of iterations, 100.
static double DEFAULT_MIN_SENSITIVITY
          The default minimum sensitivity, 10.0.
 
Fields inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner
data, keepGoing
 
Fields inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm
maxIterations
 
Fields inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm
DEFAULT_ITERATION, iteration
 
Constructor Summary
KernelBasedIterativeRegression()
          Creates a new instance of KernelBasedIterativeRegression.
KernelBasedIterativeRegression(Kernel<? super InputType> kernel)
          Creates a new KernelBasedIterativeRegression with the given kernel.
KernelBasedIterativeRegression(Kernel<? super InputType> kernel, double minSensitivity)
          Creates a new KernelBasedIterativeRegression with the given kernel.
KernelBasedIterativeRegression(Kernel<? super InputType> kernel, double minSensitivity, int maxIterations)
          Creates a new KernelBasedIterativeRegression with the given kernel and maximum number of iterations.
 
Method Summary
protected  void cleanupAlgorithm()
          Called to clean up the learning algorithm's state after learning has finished.
 KernelBasedIterativeRegression<InputType> clone()
          This makes public the clone method on the Object class and removes the exception that it throws.
 int getErrorCount()
          Gets the error count of the most recent iteration.
 Kernel<? super InputType> getKernel()
          Gets the kernel to use.
 double getMinSensitivity()
          Gets the minimum sensitivity that an example can have on the result function.
 NamedValue<Integer> getPerformance()
          Gets the performance, which is the error count on the last iteration.
 KernelScalarFunction<InputType> getResult()
          Gets the current result of the algorithm.
protected  LinkedHashMap<InputOutputPair<? extends InputType,Double>,DefaultWeightedValue<InputType>> getSupportsMap()
          Gets the mapping of examples to weight objects (support vectors).
protected  boolean initializeAlgorithm()
          Called to initialize the learning algorithm's state based on the data that is stored in the data field.
protected  void setErrorCount(int errorCount)
          Sets the error count of the most recent iteration.
 void setKernel(Kernel<? super InputType> kernel)
          Sets the kernel to use.
 void setMinSensitivity(double minSensitivity)
          Sets the minimum sensitivity that an example can have on the result function.
protected  void setResult(KernelScalarFunction<InputType> result)
          Sets the object currently being result.
protected  void setSupportsMap(LinkedHashMap<InputOutputPair<? extends InputType,Double>,DefaultWeightedValue<InputType>> supportsMap)
          Gets the mapping of examples to weight objects (support vectors).
protected  boolean step()
          Called to take a single step of the learning algorithm.
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner
getData, getKeepGoing, learn, setData, setKeepGoing, stop
 
Methods inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm
getMaxIterations, isResultValid, setMaxIterations
 
Methods inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm
addIterativeAlgorithmListener, fireAlgorithmEnded, fireAlgorithmStarted, fireStepEnded, fireStepStarted, getIteration, getListeners, removeIterativeAlgorithmListener, setIteration, setListeners
 
Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchLearner
learn
 
Methods inherited from interface gov.sandia.cognition.algorithm.AnytimeAlgorithm
getMaxIterations, setMaxIterations
 
Methods inherited from interface gov.sandia.cognition.algorithm.IterativeAlgorithm
addIterativeAlgorithmListener, getIteration, removeIterativeAlgorithmListener
 
Methods inherited from interface gov.sandia.cognition.algorithm.StoppableAlgorithm
isResultValid
 

Field Detail

DEFAULT_MAX_ITERATIONS

public static final int DEFAULT_MAX_ITERATIONS
The default maximum number of iterations, 100.

See Also:
Constant Field Values

DEFAULT_MIN_SENSITIVITY

public static final double DEFAULT_MIN_SENSITIVITY
The default minimum sensitivity, 10.0.

See Also:
Constant Field Values
Constructor Detail

KernelBasedIterativeRegression

public KernelBasedIterativeRegression()
Creates a new instance of KernelBasedIterativeRegression.


KernelBasedIterativeRegression

public KernelBasedIterativeRegression(Kernel<? super InputType> kernel)
Creates a new KernelBasedIterativeRegression with the given kernel.

Parameters:
kernel - The kernel to use.

KernelBasedIterativeRegression

public KernelBasedIterativeRegression(Kernel<? super InputType> kernel,
                                      double minSensitivity)
Creates a new KernelBasedIterativeRegression with the given kernel.

Parameters:
kernel - The kernel to use.
minSensitivity - The minimum sensitivity to errors.

KernelBasedIterativeRegression

public KernelBasedIterativeRegression(Kernel<? super InputType> kernel,
                                      double minSensitivity,
                                      int maxIterations)
Creates a new KernelBasedIterativeRegression with the given kernel and maximum number of iterations.

Parameters:
kernel - The kernel to use.
minSensitivity - The minimum sensitivity to errors.
maxIterations - The maximum number of iterations.
Method Detail

clone

public KernelBasedIterativeRegression<InputType> clone()
Description copied from class: AbstractCloneableSerializable
This makes public the clone method on the Object class and removes the exception that it throws. Its default behavior is to automatically create a clone of the exact type of object that the clone is called on and to copy all primitives but to keep all references, which means it is a shallow copy. Extensions of this class may want to override this method (but call super.clone() to implement a "smart copy". That is, to target the most common use case for creating a copy of the object. Because of the default behavior being a shallow copy, extending classes only need to handle fields that need to have a deeper copy (or those that need to be reset). Some of the methods in ObjectUtil may be helpful in implementing a custom clone method. Note: The contract of this method is that you must use super.clone() as the basis for your implementation.

Specified by:
clone in interface CloneableSerializable
Overrides:
clone in class AbstractAnytimeBatchLearner<Collection<? extends InputOutputPair<? extends InputType,Double>>,KernelScalarFunction<InputType>>
Returns:
A clone of this object.

initializeAlgorithm

protected boolean initializeAlgorithm()
Description copied from class: AbstractAnytimeBatchLearner
Called to initialize the learning algorithm's state based on the data that is stored in the data field. The return value indicates if the algorithm can be run or not based on the initialization.

Specified by:
initializeAlgorithm in class AbstractAnytimeBatchLearner<Collection<? extends InputOutputPair<? extends InputType,Double>>,KernelScalarFunction<InputType>>
Returns:
True if the learning algorithm can be run and false if it cannot.

step

protected boolean step()
Description copied from class: AbstractAnytimeBatchLearner
Called to take a single step of the learning algorithm.

Specified by:
step in class AbstractAnytimeBatchLearner<Collection<? extends InputOutputPair<? extends InputType,Double>>,KernelScalarFunction<InputType>>
Returns:
True if another step can be taken and false it the algorithm should halt.

cleanupAlgorithm

protected void cleanupAlgorithm()
Description copied from class: AbstractAnytimeBatchLearner
Called to clean up the learning algorithm's state after learning has finished.

Specified by:
cleanupAlgorithm in class AbstractAnytimeBatchLearner<Collection<? extends InputOutputPair<? extends InputType,Double>>,KernelScalarFunction<InputType>>

getKernel

public Kernel<? super InputType> getKernel()
Gets the kernel to use.

Returns:
The kernel to use.

setKernel

public void setKernel(Kernel<? super InputType> kernel)
Sets the kernel to use.

Parameters:
kernel - The kernel to use.

getResult

public KernelScalarFunction<InputType> getResult()
Description copied from interface: AnytimeAlgorithm
Gets the current result of the algorithm.

Specified by:
getResult in interface AnytimeAlgorithm<KernelScalarFunction<InputType>>
Returns:
Current result of the algorithm.

setResult

protected void setResult(KernelScalarFunction<InputType> result)
Sets the object currently being result.

Parameters:
result - The object currently being result.

getErrorCount

public int getErrorCount()
Gets the error count of the most recent iteration.

Returns:
The current error count.

setErrorCount

protected void setErrorCount(int errorCount)
Sets the error count of the most recent iteration.

Parameters:
errorCount - The current error count.

getSupportsMap

protected LinkedHashMap<InputOutputPair<? extends InputType,Double>,DefaultWeightedValue<InputType>> getSupportsMap()
Gets the mapping of examples to weight objects (support vectors).

Returns:
The mapping of examples to weight objects.

setSupportsMap

protected void setSupportsMap(LinkedHashMap<InputOutputPair<? extends InputType,Double>,DefaultWeightedValue<InputType>> supportsMap)
Gets the mapping of examples to weight objects (support vectors).

Parameters:
supportsMap - The mapping of examples to weight objects.

getMinSensitivity

public double getMinSensitivity()
Gets the minimum sensitivity that an example can have on the result function.

Returns:
The minimum sensitivity.

setMinSensitivity

public void setMinSensitivity(double minSensitivity)
Sets the minimum sensitivity that an example can have on the result function.

Parameters:
minSensitivity - The minimum sensitivity.

getPerformance

public NamedValue<Integer> getPerformance()
Gets the performance, which is the error count on the last iteration.

Specified by:
getPerformance in interface MeasurablePerformanceAlgorithm
Returns:
The performance of the algorithm.