gov.sandia.cognition.learning.algorithm.ensemble
Class BaggingRegressionLearner<InputType>

java.lang.Object
  extended by gov.sandia.cognition.util.AbstractCloneableSerializable
      extended by gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm
          extended by gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm<ResultType>
              extended by gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner<Collection<? extends InputOutputPair<? extends InputType,OutputType>>,ResultType>
                  extended by gov.sandia.cognition.learning.algorithm.AbstractAnytimeSupervisedBatchLearner<InputType,OutputType,EnsembleType>
                      extended by gov.sandia.cognition.learning.algorithm.ensemble.AbstractBaggingLearner<InputType,Double,Evaluator<? super InputType,? extends Number>,AveragingEnsemble<InputType,Evaluator<? super InputType,? extends Number>>>
                          extended by gov.sandia.cognition.learning.algorithm.ensemble.BaggingRegressionLearner<InputType>
Type Parameters:
InputType - The input type for supervised learning. Passed on to the internal learning algorithm. Also the input type for the learned ensemble.
All Implemented Interfaces:
AnytimeAlgorithm<AveragingEnsemble<InputType,Evaluator<? super InputType,? extends Number>>>, IterativeAlgorithm, StoppableAlgorithm, AnytimeBatchLearner<Collection<? extends InputOutputPair<? extends InputType,Double>>,AveragingEnsemble<InputType,Evaluator<? super InputType,? extends Number>>>, BatchLearner<Collection<? extends InputOutputPair<? extends InputType,Double>>,AveragingEnsemble<InputType,Evaluator<? super InputType,? extends Number>>>, BatchLearnerContainer<BatchLearner<? super Collection<? extends InputOutputPair<? extends InputType,Double>>,? extends Evaluator<? super InputType,? extends Number>>>, SupervisedBatchLearner<InputType,Double,AveragingEnsemble<InputType,Evaluator<? super InputType,? extends Number>>>, CloneableSerializable, Randomized, Serializable, Cloneable

@PublicationReference(title="Bagging Predictors",
                      author="Leo Breiman",
                      year=1996,
                      type=Journal,
                      publication="Machine Learning",
                      pages={123,140},
                      url="http://www.springerlink.com/index/L4780124W2874025.pdf")
public class BaggingRegressionLearner<InputType>
extends AbstractBaggingLearner<InputType,Double,Evaluator<? super InputType,? extends Number>,AveragingEnsemble<InputType,Evaluator<? super InputType,? extends Number>>>

Learns an ensemble for regression by randomly sampling with replacement (duplicates allowed) some percentage of the size of the data (defaults to 100%) on each iteration to train a new ensemble member. The random sample is referred to as a bag. Each learned ensemble member is given equal weight. The idea here is that randomly sampling from the data and learning an ensemble member that has high variance (such as a decision tree) with respect to the input data, one can improve the performance of that algorithm. By default, the algorithm runs the maxIterations number of steps to create that number of ensemble members. However, one can also use out-of-bag (OOB) error on each iteration to determine a stopping criteria. The OOB error is determined by looking at the performance of the categorizer on the examples that it has not seen.

Since:
3.3.3
Author:
Justin Basilico
See Also:
BaggingCategorizerLearner, Serialized Form

Field Summary
 
Fields inherited from class gov.sandia.cognition.learning.algorithm.ensemble.AbstractBaggingLearner
bag, dataInBag, dataList, DEFAULT_MAX_ITERATIONS, DEFAULT_PERCENT_TO_SAMPLE, ensemble, learner, percentToSample, random
 
Fields inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner
data, keepGoing
 
Fields inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm
maxIterations
 
Fields inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm
DEFAULT_ITERATION, iteration
 
Constructor Summary
BaggingRegressionLearner()
          Creates a new, empty BaggingRegressionLearner.
BaggingRegressionLearner(BatchLearner<? super Collection<? extends InputOutputPair<? extends InputType,Double>>,? extends Evaluator<? super InputType,? extends Number>> learner)
          Creates a new instance of BaggingRegressionLearner.
BaggingRegressionLearner(BatchLearner<? super Collection<? extends InputOutputPair<? extends InputType,Double>>,? extends Evaluator<? super InputType,? extends Number>> learner, int maxIterations, double percentToSample, Random random)
          Creates a new instance of BaggingRegressionLearner.
 
Method Summary
protected  void addEnsembleMember(Evaluator<? super InputType,? extends Number> member)
          Adds a new member to the ensemble.
protected  AveragingEnsemble<InputType,Evaluator<? super InputType,? extends Number>> createInitialEnsemble()
          Create the initial, empty ensemble for the algorithm to use.
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.ensemble.AbstractBaggingLearner
cleanupAlgorithm, fillBag, getBag, getDataInBag, getDataList, getEnsemble, getLearner, getPercentToSample, getRandom, getResult, initializeAlgorithm, setBag, setDataInBag, setDataList, setEnsemble, setLearner, setPercentToSample, setRandom, step
 
Methods inherited from class gov.sandia.cognition.learning.algorithm.AbstractAnytimeBatchLearner
clone, getData, getKeepGoing, learn, setData, setKeepGoing, stop
 
Methods inherited from class gov.sandia.cognition.algorithm.AbstractAnytimeAlgorithm
getMaxIterations, isResultValid, setMaxIterations
 
Methods inherited from class gov.sandia.cognition.algorithm.AbstractIterativeAlgorithm
addIterativeAlgorithmListener, fireAlgorithmEnded, fireAlgorithmStarted, fireStepEnded, fireStepStarted, getIteration, getListeners, removeIterativeAlgorithmListener, setIteration, setListeners
 
Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface gov.sandia.cognition.learning.algorithm.BatchLearner
learn
 
Methods inherited from interface gov.sandia.cognition.util.CloneableSerializable
clone
 
Methods inherited from interface gov.sandia.cognition.algorithm.AnytimeAlgorithm
getMaxIterations, setMaxIterations
 
Methods inherited from interface gov.sandia.cognition.algorithm.IterativeAlgorithm
addIterativeAlgorithmListener, getIteration, removeIterativeAlgorithmListener
 
Methods inherited from interface gov.sandia.cognition.algorithm.StoppableAlgorithm
isResultValid
 

Constructor Detail

BaggingRegressionLearner

public BaggingRegressionLearner()
Creates a new, empty BaggingRegressionLearner.


BaggingRegressionLearner

public BaggingRegressionLearner(BatchLearner<? super Collection<? extends InputOutputPair<? extends InputType,Double>>,? extends Evaluator<? super InputType,? extends Number>> learner)
Creates a new instance of BaggingRegressionLearner.

Parameters:
learner - The learner to use to create the categorizer on each iteration.

BaggingRegressionLearner

public BaggingRegressionLearner(BatchLearner<? super Collection<? extends InputOutputPair<? extends InputType,Double>>,? extends Evaluator<? super InputType,? extends Number>> learner,
                                int maxIterations,
                                double percentToSample,
                                Random random)
Creates a new instance of BaggingRegressionLearner.

Parameters:
learner - The learner to use to create the regression function on each iteration.
maxIterations - The maximum number of iterations to run for, which is also the number of learners to create.
percentToSample - The percentage of the total size of the data to sample on each iteration. Must be positive.
random - The random number generator to use.
Method Detail

createInitialEnsemble

protected AveragingEnsemble<InputType,Evaluator<? super InputType,? extends Number>> createInitialEnsemble()
Description copied from class: AbstractBaggingLearner
Create the initial, empty ensemble for the algorithm to use.

Specified by:
createInitialEnsemble in class AbstractBaggingLearner<InputType,Double,Evaluator<? super InputType,? extends Number>,AveragingEnsemble<InputType,Evaluator<? super InputType,? extends Number>>>
Returns:
A new ensemble for the algorithm to use.

addEnsembleMember

protected void addEnsembleMember(Evaluator<? super InputType,? extends Number> member)
Description copied from class: AbstractBaggingLearner
Adds a new member to the ensemble.

Specified by:
addEnsembleMember in class AbstractBaggingLearner<InputType,Double,Evaluator<? super InputType,? extends Number>,AveragingEnsemble<InputType,Evaluator<? super InputType,? extends Number>>>
Parameters:
member - The new member to add to the ensemble.