gov.sandia.cognition.statistics.bayesian
Class BayesianRobustLinearRegression

java.lang.Object
  extended by gov.sandia.cognition.util.AbstractCloneableSerializable
      extended by gov.sandia.cognition.statistics.bayesian.BayesianRobustLinearRegression
All Implemented Interfaces:
BatchLearner<Collection<? extends InputOutputPair<? extends Vectorizable,Double>>,MultivariateGaussianInverseGammaDistribution>, BayesianEstimator<InputOutputPair<? extends Vectorizable,Double>,Vector,MultivariateGaussianInverseGammaDistribution>, BayesianRegression<Double,MultivariateGaussianInverseGammaDistribution>, CloneableSerializable, Serializable, Cloneable
Direct Known Subclasses:
BayesianRobustLinearRegression.IncrementalEstimator

@PublicationReferences(references={@PublicationReference(author="Christopher M. Bishop",title="Pattern Recognition and Machine Learning",type=Book,year=2006,pages={152,159}),@PublicationReference(author="Jan Drugowitsch",title="Bayesian Linear Regression",type=Misc,year=2009,url="http://www.bcs.rochester.edu/people/jdrugowitsch/code/bayes_linear_notes_0.1.1.pdf")})
public class BayesianRobustLinearRegression
extends AbstractCloneableSerializable
implements BayesianRegression<Double,MultivariateGaussianInverseGammaDistribution>

Computes a Bayesian linear estimator for a given feature function given a set of InputOutputPair observed values. This regression algorithm assumes that the conditional distribution of observations is a zero-mean Gaussian with unknown variance, distributed according to the standard (conjugate prior) inverse-Gamma distribution. The prior distribution of weights is a multivariate Gaussian. Given a training set, the algorithm computes the posterior of the weights given the training set. The system perform robust regression because the predictive distribution of future data is a Student-t distribution, resulting from the assumption that the variance of the outputs is unknown. Consequently, the algorithm is more robust against outliers than fix-variance regression.

Since:
3.0
Author:
Kevin R. Dixon
See Also:
Serialized Form

Nested Class Summary
static class BayesianRobustLinearRegression.IncrementalEstimator
          Incremental estimator for BayesianRobustLinearRegression
 class BayesianRobustLinearRegression.PredictiveDistribution
          Predictive distribution of future data given the posterior of the weights given the data.
 
Field Summary
static double DEFAULT_WEIGHT_VARIANCE
          Default weight variance, 1.0.
 
Constructor Summary
BayesianRobustLinearRegression(int dimensionality)
          Creates a new instance of BayesianLinearRegression
BayesianRobustLinearRegression(InverseGammaDistribution outputVariance, MultivariateGaussian weightPrior)
          Creates a new instance of BayesianRobustLinearRegression
 
Method Summary
 BayesianRobustLinearRegression clone()
          This makes public the clone method on the Object class and removes the exception that it throws.
 UnivariateGaussian createConditionalDistribution(Vectorizable input, Vector weights)
          Creates the distribution from which the outputs are generated, given the weights and the input to consider.
 BayesianRobustLinearRegression.PredictiveDistribution createPredictiveDistribution(MultivariateGaussianInverseGammaDistribution posterior)
          Creates the predictive distribution of outputs given the weight posterior
 InverseGammaDistribution getOutputVariance()
          Getter for outputVariance
 MultivariateGaussian getWeightPrior()
          Getter for weightPrior
 MultivariateGaussianInverseGammaDistribution learn(Collection<? extends InputOutputPair<? extends Vectorizable,Double>> data)
          The learn method creates an object of ResultType using data of type DataType, using some form of "learning" algorithm.
 void setOutputVariance(InverseGammaDistribution outputVariance)
          Setter for outputVariance
 void setWeightPrior(MultivariateGaussian weightPrior)
          Setter for weightPrior
 
Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

DEFAULT_WEIGHT_VARIANCE

public static final double DEFAULT_WEIGHT_VARIANCE
Default weight variance, 1.0.

See Also:
Constant Field Values
Constructor Detail

BayesianRobustLinearRegression

public BayesianRobustLinearRegression(int dimensionality)
Creates a new instance of BayesianLinearRegression

Parameters:
dimensionality - Sets up the parameters (except featureMap) for the given dimensionality of objects in feature space.

BayesianRobustLinearRegression

public BayesianRobustLinearRegression(InverseGammaDistribution outputVariance,
                                      MultivariateGaussian weightPrior)
Creates a new instance of BayesianRobustLinearRegression

Parameters:
outputVariance - Distribution of the output (measurement) variance
weightPrior - Prior distribution of the weights, typically a zero-mean, diagonal-variance distribution.
Method Detail

clone

public BayesianRobustLinearRegression clone()
Description copied from class: AbstractCloneableSerializable
This makes public the clone method on the Object class and removes the exception that it throws. Its default behavior is to automatically create a clone of the exact type of object that the clone is called on and to copy all primitives but to keep all references, which means it is a shallow copy. Extensions of this class may want to override this method (but call super.clone() to implement a "smart copy". That is, to target the most common use case for creating a copy of the object. Because of the default behavior being a shallow copy, extending classes only need to handle fields that need to have a deeper copy (or those that need to be reset). Some of the methods in ObjectUtil may be helpful in implementing a custom clone method. Note: The contract of this method is that you must use super.clone() as the basis for your implementation.

Specified by:
clone in interface CloneableSerializable
Overrides:
clone in class AbstractCloneableSerializable
Returns:
A clone of this object.

getWeightPrior

public MultivariateGaussian getWeightPrior()
Getter for weightPrior

Returns:
Prior distribution of the weights, typically a zero-mean, diagonal-variance distribution.

setWeightPrior

public void setWeightPrior(MultivariateGaussian weightPrior)
Setter for weightPrior

Parameters:
weightPrior - Prior distribution of the weights, typically a zero-mean, diagonal-variance distribution.

learn

public MultivariateGaussianInverseGammaDistribution learn(Collection<? extends InputOutputPair<? extends Vectorizable,Double>> data)
Description copied from interface: BatchLearner
The learn method creates an object of ResultType using data of type DataType, using some form of "learning" algorithm.

Specified by:
learn in interface BatchLearner<Collection<? extends InputOutputPair<? extends Vectorizable,Double>>,MultivariateGaussianInverseGammaDistribution>
Parameters:
data - The data that the learning algorithm will use to create an object of ResultType.
Returns:
The object that is created based on the given data using the learning algorithm.

getOutputVariance

public InverseGammaDistribution getOutputVariance()
Getter for outputVariance

Returns:
Distribution of the output (measurement) variance

setOutputVariance

public void setOutputVariance(InverseGammaDistribution outputVariance)
Setter for outputVariance

Parameters:
outputVariance - Distribution of the output (measurement) variance

createConditionalDistribution

public UnivariateGaussian createConditionalDistribution(Vectorizable input,
                                                        Vector weights)
Creates the distribution from which the outputs are generated, given the weights and the input to consider.

Specified by:
createConditionalDistribution in interface BayesianRegression<Double,MultivariateGaussianInverseGammaDistribution>
Parameters:
input - Input to condition on
weights - Weights that determine the mean
Returns:
Conditional distribution from which outputs are generated.

createPredictiveDistribution

public BayesianRobustLinearRegression.PredictiveDistribution createPredictiveDistribution(MultivariateGaussianInverseGammaDistribution posterior)
Description copied from interface: BayesianRegression
Creates the predictive distribution of outputs given the weight posterior

Specified by:
createPredictiveDistribution in interface BayesianRegression<Double,MultivariateGaussianInverseGammaDistribution>
Parameters:
posterior - Posterior distribution of weights.
Returns:
Predictive distribution of outputs given the posterior.