Class ShafferStaticCorrection

  extended by gov.sandia.cognition.util.AbstractCloneableSerializable
      extended by gov.sandia.cognition.statistics.method.AbstractMultipleHypothesisComparison<Collection<? extends Number>,StatisticType>
          extended by gov.sandia.cognition.statistics.method.AbstractPairwiseMultipleHypothesisComparison<ShafferStaticCorrection.Statistic>
              extended by gov.sandia.cognition.statistics.method.ShafferStaticCorrection
All Implemented Interfaces:
MultipleHypothesisComparison<Collection<? extends Number>>, CloneableSerializable, Serializable, Cloneable

@PublicationReferences(references={@PublicationReference(author="Juliet Popper Shaffer",title="Modified Sequentially Rejective Multiple Test Procedures",type=Journal,publication="Journal of the American Statistical Association",year=1986,url=""),@PublicationReference(author="Juliet Popper Shaffer",title="Multiple Hypothesis Testing",type=Journal,publication="Annual Review of Psychology",year=1995,url=""),@PublicationReference(author={"Salvador Garcia","Francisco Herrera"},title="An Extension on \"Statistical Comparisons of Classi\ufb01ers over Multiple Data Sets\" for all Pairwise Comparisons",type=Journal,publication="Journal of Machine Learning Research",year=2008,url="")})
public class ShafferStaticCorrection
extends AbstractPairwiseMultipleHypothesisComparison<ShafferStaticCorrection.Statistic>

The Shaffer Static Correction uses logical relationships to tighten up the Bonferroni/Sidak corrections when performing pairwise multiple hypothesis comparisons. This is uniformly tighter bound than the Bonferroni/Sidak values and also uniformly tighter than the Holm correction. The original algorithm proposed by Shaffer appears to grow super exponentially as O(2^(N^2)), where N is the number of treatments. We have pre-computed various quantities and used caching to minimize the amount of repeated recursion and it appears that the runtime grows as O(N^4), where N is the number of treatments. Since there are N(N-1)/2 comparisons, this quantity is quadratic in the number of comparisons. This means the computation is reasonable for N=90 (4005 comparisons). However, the algorithm seems to slow down significantly above N=100 (4950 comparisons) or so. This implementation uses the slightly tighter Sidak correction, as opposed to the standard Bonferroni correction.

For example, if you have three hypothesis you are testing, then there is no way that there can be 2 true hypotheses. (Because u1=u2 and u2=u3 and therefore u1=u3.) With this logic, Shaffer Static correction can greatly reduce the false-negative rate while not impacting the false-discovery rate.

Kevin R. Dixon
See Also:
Serialized Form

Nested Class Summary
static class ShafferStaticCorrection.Statistic
          Test statistic from the Shaffer static multiple-comparison test
Field Summary
Fields inherited from class gov.sandia.cognition.statistics.method.AbstractPairwiseMultipleHypothesisComparison
Fields inherited from interface gov.sandia.cognition.statistics.method.MultipleHypothesisComparison
Constructor Summary
          Default constructor
ShafferStaticCorrection(NullHypothesisEvaluator<Collection<? extends Number>> pairwiseTest)
          Creates a new instance of BonferroniCorrection
Method Summary
 ShafferStaticCorrection clone()
          This makes public the clone method on the Object class and removes the exception that it throws.
 ShafferStaticCorrection.Statistic evaluateNullHypotheses(Collection<? extends Collection<? extends Number>> data, double uncompensatedAlpha)
          Evaluates the null hypotheses associated with the given collection of data.
Methods inherited from class gov.sandia.cognition.statistics.method.AbstractPairwiseMultipleHypothesisComparison
getPairwiseTest, setPairwiseTest
Methods inherited from class gov.sandia.cognition.statistics.method.AbstractMultipleHypothesisComparison
Methods inherited from class java.lang.Object
equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail


public ShafferStaticCorrection()
Default constructor


public ShafferStaticCorrection(NullHypothesisEvaluator<Collection<? extends Number>> pairwiseTest)
Creates a new instance of BonferroniCorrection

pairwiseTest - Confidence test used for pair-wise null-hypothesis tests.
Method Detail


public ShafferStaticCorrection clone()
Description copied from class: AbstractCloneableSerializable
This makes public the clone method on the Object class and removes the exception that it throws. Its default behavior is to automatically create a clone of the exact type of object that the clone is called on and to copy all primitives but to keep all references, which means it is a shallow copy. Extensions of this class may want to override this method (but call super.clone() to implement a "smart copy". That is, to target the most common use case for creating a copy of the object. Because of the default behavior being a shallow copy, extending classes only need to handle fields that need to have a deeper copy (or those that need to be reset). Some of the methods in ObjectUtil may be helpful in implementing a custom clone method. Note: The contract of this method is that you must use super.clone() as the basis for your implementation.

Specified by:
clone in interface CloneableSerializable
clone in class AbstractPairwiseMultipleHypothesisComparison<ShafferStaticCorrection.Statistic>
A clone of this object.


public ShafferStaticCorrection.Statistic evaluateNullHypotheses(Collection<? extends Collection<? extends Number>> data,
                                                                double uncompensatedAlpha)
Description copied from interface: MultipleHypothesisComparison
Evaluates the null hypotheses associated with the given collection of data.

Specified by:
evaluateNullHypotheses in interface MultipleHypothesisComparison<Collection<? extends Number>>
Specified by:
evaluateNullHypotheses in class AbstractMultipleHypothesisComparison<Collection<? extends Number>,ShafferStaticCorrection.Statistic>
data - Data from each treatment to consider
uncompensatedAlpha - Uncompensated alpha (p-value threshold) for the multiple comparison test, must be [0,1]
Statistic that summarizes the multiple comparison test