gov.sandia.cognition.learning.experiment
Class OnlineLearnerValidationExperiment<DataType,LearnedType,StatisticType,SummaryType>

java.lang.Object
  extended by gov.sandia.cognition.util.AbstractCloneableSerializable
      extended by gov.sandia.cognition.learning.experiment.AbstractLearningExperiment
          extended by gov.sandia.cognition.learning.experiment.OnlineLearnerValidationExperiment<DataType,LearnedType,StatisticType,SummaryType>
Type Parameters:
DataType - The type of the data to perform the experiment with. It will be used as input into the learning algorithm.
LearnedType - The type of the output produced by the learning algorithm whose performance will be evaluated on each data item.
StatisticType - The type of the statistic generated by the performance evaluator on the learned object for each data item. It is created by passing the learned object plus the data item into the performance evaluator.
SummaryType - The type produced by the summarizer at the end of the experiment from a collection of the given statistics (one for each item). This represents the performance result for the learning algorithm for the whole experiment.
All Implemented Interfaces:
LearningExperiment, PerformanceEvaluator<IncrementalLearner<? super DataType,LearnedType>,Collection<? extends DataType>,SummaryType>, CloneableSerializable, Serializable, Cloneable

public class OnlineLearnerValidationExperiment<DataType,LearnedType,StatisticType,SummaryType>
extends AbstractLearningExperiment
implements PerformanceEvaluator<IncrementalLearner<? super DataType,LearnedType>,Collection<? extends DataType>,SummaryType>, Serializable

Implements an experiment where an incremental supervised machine learning algorithm is evaluated by applying it to a set of data by successively testing on each item and then training on it.

Since:
3.0
Author:
Justin Basilico
See Also:
Serialized Form

Field Summary
protected  int numTrials
          The number of trials in the experiment, which is the number of folds in the experiment.
protected  PerformanceEvaluator<? super LearnedType,? super Collection<? extends DataType>,? extends StatisticType> performanceEvaluator
          The evaluator to use to compute the performance of the learned object on each fold.
protected  ArrayList<StatisticType> statistics
          The performance evaluations made during the experiment.
protected  Summarizer<? super StatisticType,? extends SummaryType> summarizer
          The summarizer for summarizing the result of the performance evaluator from all the folds.
protected  SummaryType summary
          The summary of the performance evaluations made at the end of the experiment.
 
Fields inherited from class gov.sandia.cognition.learning.experiment.AbstractLearningExperiment
listeners
 
Constructor Summary
OnlineLearnerValidationExperiment()
          Creates a new instance of IncrementalLearnerValidationExperiment.
OnlineLearnerValidationExperiment(PerformanceEvaluator<? super LearnedType,? super Collection<? extends DataType>,? extends StatisticType> performanceEvaluator, Summarizer<? super StatisticType,? extends SummaryType> summarizer)
          Creates a new instance of IncrementalLearnerValidationExperiment.
 
Method Summary
 SummaryType evaluatePerformance(IncrementalLearner<? super DataType,LearnedType> learner, Collection<? extends DataType> data)
          Performs the experiment.
 int getNumTrials()
          Gets the number of trials.
 PerformanceEvaluator<? super LearnedType,? super Collection<? extends DataType>,? extends StatisticType> getPerformanceEvaluator()
          Gets the performance evaluator to apply to each fold.
 ArrayList<StatisticType> getStatistics()
          Gets the performance evaluations for the trials of the experiment.
 Summarizer<? super StatisticType,? extends SummaryType> getSummarizer()
          Gets the summarizer of the performance evaluations.
 SummaryType getSummary()
          Gets the summary of the experiment.
protected  void setNumTrials(int numTrials)
          Sets the number of trials.
 void setPerformanceEvaluator(PerformanceEvaluator<? super LearnedType,? super Collection<? extends DataType>,? extends StatisticType> performanceEvaluator)
          Sets the performance evaluator to apply to each fold.
protected  void setStatistics(ArrayList<StatisticType> statistics)
          Sets the performance evaluations for the trials of the experiment.
 void setSummarizer(Summarizer<? super StatisticType,? extends SummaryType> summarizer)
          Sets the summarizer of the performance evaluations.
protected  void setSummary(SummaryType summary)
          Sets the summary of the experiment.
 
Methods inherited from class gov.sandia.cognition.learning.experiment.AbstractLearningExperiment
addListener, fireExperimentEnded, fireExperimentStarted, fireTrialEnded, fireTrialStarted, getListeners, removeListener, setListeners
 
Methods inherited from class gov.sandia.cognition.util.AbstractCloneableSerializable
clone
 
Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 
Methods inherited from interface gov.sandia.cognition.util.CloneableSerializable
clone
 

Field Detail

performanceEvaluator

protected PerformanceEvaluator<? super LearnedType,? super Collection<? extends DataType>,? extends StatisticType> performanceEvaluator
The evaluator to use to compute the performance of the learned object on each fold.


summarizer

protected Summarizer<? super StatisticType,? extends SummaryType> summarizer
The summarizer for summarizing the result of the performance evaluator from all the folds.


numTrials

protected int numTrials
The number of trials in the experiment, which is the number of folds in the experiment.


statistics

protected ArrayList<StatisticType> statistics
The performance evaluations made during the experiment.


summary

protected SummaryType summary
The summary of the performance evaluations made at the end of the experiment.

Constructor Detail

OnlineLearnerValidationExperiment

public OnlineLearnerValidationExperiment()
Creates a new instance of IncrementalLearnerValidationExperiment.


OnlineLearnerValidationExperiment

public OnlineLearnerValidationExperiment(PerformanceEvaluator<? super LearnedType,? super Collection<? extends DataType>,? extends StatisticType> performanceEvaluator,
                                         Summarizer<? super StatisticType,? extends SummaryType> summarizer)
Creates a new instance of IncrementalLearnerValidationExperiment.

Parameters:
performanceEvaluator - The evaluator to use to compute the performance of the learned object on each fold.
summarizer - The summarizer for summarizing the result of the performance evaluator from all the folds.
Method Detail

evaluatePerformance

public SummaryType evaluatePerformance(IncrementalLearner<? super DataType,LearnedType> learner,
                                       Collection<? extends DataType> data)
Performs the experiment.

Specified by:
evaluatePerformance in interface PerformanceEvaluator<IncrementalLearner<? super DataType,LearnedType>,Collection<? extends DataType>,SummaryType>
Parameters:
data - The data to use.
learner - The learner to perform the experiment on.
Returns:
The summary of the experiment.

getPerformanceEvaluator

public PerformanceEvaluator<? super LearnedType,? super Collection<? extends DataType>,? extends StatisticType> getPerformanceEvaluator()
Gets the performance evaluator to apply to each fold.

Returns:
The performance evaluator to apply to each fold.

setPerformanceEvaluator

public void setPerformanceEvaluator(PerformanceEvaluator<? super LearnedType,? super Collection<? extends DataType>,? extends StatisticType> performanceEvaluator)
Sets the performance evaluator to apply to each fold.

Parameters:
performanceEvaluator - The performance evaluator to apply to each fold.

getSummarizer

public Summarizer<? super StatisticType,? extends SummaryType> getSummarizer()
Gets the summarizer of the performance evaluations.

Returns:
The summarizer of the performance evaluations.

setSummarizer

public void setSummarizer(Summarizer<? super StatisticType,? extends SummaryType> summarizer)
Sets the summarizer of the performance evaluations.

Parameters:
summarizer - The summarizer of the performance evaluations.

getNumTrials

public int getNumTrials()
Gets the number of trials. Will be equal to the number of data points that the experiment is being run over.

Specified by:
getNumTrials in interface LearningExperiment
Returns:
The number of trials.

setNumTrials

protected void setNumTrials(int numTrials)
Sets the number of trials.

Parameters:
numTrials - The number of trials.

getStatistics

public ArrayList<StatisticType> getStatistics()
Gets the performance evaluations for the trials of the experiment.

Returns:
The performance evaluations for the trials of the experiment.

setStatistics

protected void setStatistics(ArrayList<StatisticType> statistics)
Sets the performance evaluations for the trials of the experiment.

Parameters:
statistics - The performance evaluations for the trials of the experiment.

getSummary

public SummaryType getSummary()
Gets the summary of the experiment.

Returns:
The summary of the experiment.

setSummary

protected void setSummary(SummaryType summary)
Sets the summary of the experiment.

Parameters:
summary - The summary of the experiment.