Overview | Package | Class | Tree | Deprecated | Index | Help
| SSJ V. 2.1.1.
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

## umontreal.iro.lecuyer.gof
Class GofStat

    java.lang.Object
      umontreal.iro.lecuyer.gof.GofStat

    public class GofStat

extends Object

THIS CLASS WILL BE REWRITTEN AND SIMPLIFIED IN THE NEAR FUTURE. This class provides methods to compute several types of EDF goodness-of-fit test statistics and to apply certain transformations to a set of observations. This includes the probability integral transformation Ui = F(Xi), as well as the power ratio and iterated spacings transformations. Here, U(0),..., U(N-1) stand for N observations U0,..., UN-1 sorted by increasing order, where 0 <= Ui <= 1.

Note: This class uses the Colt library.

Nested Class Summary
`static class` | `GofStat.OutcomeCategoriesChi2` This class helps managing the partitions of possible outcomes into categories for applying chi-square tests.

Field Summary
`static double` | `EPSILONAD`

Method Summary
`static double` | `andersonDarling(DoubleArrayList sortedData)` Computes and returns the Anderson-Darling statistic AN2.
`static double` | `chi2(double[] nbExp, int[] count, int smin, int smax)` Computes and returns the chi-square statistic for the observations oi in `count[smin...smax]`, for which the corresponding expected values ei are in `nbExp[smin...smax]`.
`static double` | `chi2(IntArrayList data, DiscreteDistributionInt dist, int smin, int smax, double minExp, int[] m)` Computes and returns the chi-square statistic for the observations stored in `data`, assuming that these observations follow the discrete distribution `dist`.
`static double` | `chi2Equal(DoubleArrayList data)` Equivalent to `chi2Equal (data, 10)`.
`static double` | `chi2Equal(DoubleArrayList data, double minExp)` Computes the chi-square statistic for a continuous distribution.
`static double` | `chi2Equal(double nbExp, int[] count, int smin, int smax)` Similar to `chi2`, except that the expected number of observations per category is assumed to be the same for all categories, and equal to `nbExp`.
`static double` | `cramerVonMises(DoubleArrayList sortedData)` Computes and returns the Cramér-von Mises statistic WN2.
`static void` | `diff(DoubleArrayList sortedData, DoubleArrayList spacings, int n1, int n2, double a, double b)` Same as method `diff``(IntArrayList,IntArrayList,int,int,int,int)`, but for the continuous case.
`static void` | `diff(IntArrayList sortedData, IntArrayList spacings, int n1, int n2, int a, int b)` Assumes that the real-valued observations U0,..., UN-1 contained in `sortedData` are already sorted in increasing order and computes the differences between the successive observations.
`static void` | `iterateSpacings(DoubleArrayList data, DoubleArrayList spacings)` Applies one iteration of the iterated spacings transformation.
`static double[]` | `kolmogorovSmirnov(DoubleArrayList sortedData)` Computes the Kolmogorov-Smirnov (KS) test statistics DN+, DN-, and DN.
`static double[]` | `kolmogorovSmirnovJumpOne(DoubleArrayList sortedData, double a)` Compute the KS statistics DN+(a) and DN-(a) defined in the description of the method `FDist.kolmogorovSmirnovPlusJumpOne`, assuming that F is the uniform distribution over [0, 1] and that U(1),..., U(N) are in `sortedData`.
`static double` | `pDisc(double pL, double pR)` Computes a variant of the p-value p whenever a test statistic has a discrete probability distribution.
`static void` | `powerRatios(DoubleArrayList sortedData)` Applies the power ratios transformation W.
`static int` | `scan(DoubleArrayList sortedData, double d)` Computes and returns the scan statistic SN(d ), defined in `FBar.scan`.
`static DoubleArrayList` | `unifTransform(DoubleArrayList data, ContinuousDistribution dist)` Applies the transformation Ui = F(Vi) for i = 0, 1,…, N - 1, where F is a continuous distribution function, and returns the result as an array of length N.
`static DoubleArrayList` | `unifTransform(DoubleArrayList data, DiscreteDistribution dist)` Applies the transformation Ui = F(Vi) for i = 0, 1,…, N - 1, where F is a discrete distribution function, and returns the result as an array of length N.
`static double` | `watsonG(DoubleArrayList sortedData)` Computes and returns the Watson statistic GN.
`static double` | `watsonU(DoubleArrayList sortedData)` Computes and returns the Watson statistic UN2.

Methods inherited from class java.lang.Object
`equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

Field Detail

### EPSILONAD

    public static double EPSILONAD

Method Detail

### unifTransform

    public static DoubleArrayList unifTransform(DoubleArrayList data,
                                                ContinuousDistribution dist)

  Applies the transformation Ui = F(Vi) for i = 0, 1,…, N - 1, where F is a continuous distribution function, and returns the result as an array of length N. V represents the N observations contained in `data`, and U, the returned transformed observations. If `data` contains random variables from the distribution function `dist`, then the result will contain uniform random variables over [0, 1].

  Parameters:
    `data` - array of observations to be transformed
    `dist` - assumed distribution of the observations
  Returns:
    the array of transformed observations

### unifTransform

    public static DoubleArrayList unifTransform(DoubleArrayList data,
                                                DiscreteDistribution dist)

  Applies the transformation Ui = F(Vi) for i = 0, 1,…, N - 1, where F is a discrete distribution function, and returns the result as an array of length N. V represents the N observations contained in `data`, and U, the returned transformed observations.

  Note: If V are the values of random variables with distribution function `dist`, then the result will contain the values of discrete random variables distributed over the set of values taken by `dist`, not uniform random variables over [0, 1].

  Parameters:
    `data` - array of observations to be transformed
    `dist` - assumed distribution of the observations
  Returns:
    the array of transformed observations

### diff

    public static void diff(IntArrayList sortedData,
                            IntArrayList spacings,
                            int n1,
                            int n2,
                            int a,
                            int b)

  Assumes that the real-valued observations U0,..., UN-1 contained in `sortedData` are already sorted in increasing order and computes the differences between the successive observations. Let D be the differences returned in `spacings`. The difference Ui - Ui-1 is put in Di for `n1 < i <= n2`, whereas Un1 - a is put into Dn1 and b - Un2 is put into Dn2+1. The number of observations must be greater or equal than `n2`, we must have `n1 < n2`, and `n1` and `n2` are greater than 0. The size of `spacings` will be at least N + 1 after the call returns.

  Parameters:
    `sortedData` - array of sorted observations
    `spacings` - pointer to an array object that will be filled with spacings
    `n1` - starting index, in `sortedData`, of the processed observations
    `n2` - ending index, in `sortedData` of the processed observations
    `a` - minimum value of the observations
    `b` - maximum value of the observations

### diff

    public static void diff(DoubleArrayList sortedData,
                            DoubleArrayList spacings,
                            int n1,
                            int n2,
                            double a,
                            double b)

  Same as method `diff``(IntArrayList,IntArrayList,int,int,int,int)`, but for the continuous case.

  Parameters:
    `sortedData` - array of sorted observations
    `spacings` - pointer to an array object that will be filled with spacings
    `n1` - starting index of the processed observations in `sortedData`
    `n2` - ending index, in `sortedData` of the processed observations
    `a` - minimum value of the observations
    `b` - maximum value of the observations

### iterateSpacings

    public static void iterateSpacings(DoubleArrayList data,
                                       DoubleArrayList spacings)

  Applies one iteration of the iterated spacings transformation. Let U be the N observations contained into `data`, and let S be the spacings contained into `spacings`, Assumes that S[0..N] contains the spacings between N real numbers U0,..., UN-1 in the interval [0, 1]. These spacings are defined by

  Si = U(i) - U(i-1), 1 <= i < N,

  where U(0) = 0, U(N-1) = 1, and U(0),..., U(N-1), are the Ui sorted in increasing order. These spacings may have been obtained by calling `diff`. This method transforms the spacings into new spacings: it sorts S0,..., SN to obtain S(0) <= S(1) <= S(2) <= ... <= S(N), computes the weighted differences

  S0 | = | (N + 1)S(0), |
  S1 | = | N(S(1) - S(0)), |
  S2 | = | (N - 1)(S(2) - S(1)), |
  | ... |  |
  SN | = | S(N) - S(N-1), |

  and computes Vi = S0 + S1 + ... + Si for 0 <= i < N. It then returns S0,..., SN in `S[0..N]` and V1,..., VN in `V[1..N]`.

  Under the assumption that the Ui are i.i.d. U(0, 1), the new Si can be considered as a new set of spacings having the same distribution as the original spacings, and the Vi are a new sample of i.i.d. U(0, 1) random variables, sorted by increasing order.

  This transformation is useful to detect clustering in a data set: A pair of observations that are close to each other is transformed into an observation close to zero. A data set with unusually clustered observations is thus transformed to a data set with an accumulation of observations near zero, which is easily detected by the Anderson-Darling GOF test.

  Parameters:
    `data` - array of observations
    `spacings` - spacings between the observations, will be filled with the new spacings

### powerRatios

    public static void powerRatios(DoubleArrayList sortedData)

  Applies the power ratios transformation W. Let U be the N observations contained into `sortedData`. Assumes that U contains N real numbers U(0),..., U(N-1) from the interval [0, 1], already sorted in increasing order, and computes the transformations:

  U'i = (U(i)/U(i+1))i+1, i = 0,..., N - 1,

  with U(N) = 1. These U'i are sorted in increasing order and put back in `U[1...N]`. If the U(i) are i.i.d. U(0, 1) sorted by increasing order, then the U'i are also i.i.d. U(0, 1).

  This transformation is useful to detect clustering, as explained in `iterateSpacings`, except that here a pair of observations close to each other is transformed into an observation close to 1. An accumulation of observations near 1 is also easily detected by the Anderson-Darling GOF test.

  Parameters:
    `sortedData` - sorted array of real-valued observations in the interval [0, 1] that will be overwritten with the transformed observations

### chi2

    public static double chi2(double[] nbExp,
                              int[] count,
                              int smin,
                              int smax)

  Computes and returns the chi-square statistic for the observations oi in `count[smin...smax]`, for which the corresponding expected values ei are in `nbExp[smin...smax]`. Assuming that i goes from 1 to k, where k = `smax-smin+1` is the number of categories, the chi-square statistic is defined as

  X2 = ∑i=1k(oi - ei)2/ei.

  Under the hypothesis that the ei are the correct expectations and if these ei are large enough, X2 follows approximately the chi-square distribution with k - 1 degrees of freedom. If some of the ei are too small, one can use `OutcomeCategoriesChi2` to regroup categories.

  Parameters:
    `nbExp` - number of expected observations in each category (or interval)
    `count` - number of counted observations in each category
    `smin` - index of the first valid data in `count` and `nbExp`
    `smax` - index of the last valid data in `count` and `nbExp`
  Returns:
    the X2 statistic

### chi2

    public static double chi2(IntArrayList data,
                              DiscreteDistributionInt dist,
                              int smin,
                              int smax,
                              double minExp,
                              int[] m)

  Computes and returns the chi-square statistic for the observations stored in `data`, assuming that these observations follow the discrete distribution `dist`. For `dist`, we assume that there is one set S = {a, a + 1,..., b - 1, b}, where a < b and a >= 0, for which p(s) > 0 if s∈S and p(s) = 0 otherwise.

  Generally, it is not possible to divide the integers in intervals satisfying nP(a0 <= s < a1) = nP(a1 <= s < a2) = ... = nP(aj-1 <= s < aj) for a discrete distribution, where n is the sample size, i.e., the number of observations stored into `data`. To perform a general chi-square test, the method starts from `smin` and finds the first non-negligible probability p(s) >= ε, where ε = `DiscreteDistributionInt.EPSILON`. It uses `smax` to allocate an array storing the number of expected observations (np(s)) for each s >= `smin`. Starting from s = `smin`, the np(s) terms are computed and the allocated array grows if required until a negligible probability term is found. This gives the number of expected elements for each category, where an outcome category corresponds here to an interval in which sample observations could lie. The categories are regrouped to have at least `minExp` observations per category. If `m` is not `null`, the first element of the array will be set to the number of categories after regrouping. The method then counts the number of samples in each categories and calls `chi2` to get the chi-square test statistic. We usually choose `minExp` = 10.

  Parameters:
    `data` - observations, not necessarily sorted
    `dist` - assumed probability distribution
    `smin` - estimated minimum value of s for which p(s) > 0
    `smax` - estimated maximum value of s for which p(s) > 0
    `minExp` - minimum number of expected observations in each interval
    `m` - one-element array that will be filled with the number of categories after regrouping
  Returns:
    the chi-square statistic for a discrete distribution

### chi2Equal

    public static double chi2Equal(double nbExp,
                                   int[] count,
                                   int smin,
                                   int smax)

  Similar to `chi2`, except that the expected number of observations per category is assumed to be the same for all categories, and equal to `nbExp`.

  Parameters:
    `nbExp` - number of expected observations in each category (or interval)
    `count` - number of counted observations in each category
    `smin` - index of the first valid data in `count` and `nbExp`
    `smax` - index of the last valid data in `count` and `nbExp`
  Returns:
    the X2 statistic

### chi2Equal

    public static double chi2Equal(DoubleArrayList data,
                                   double minExp)

  Computes the chi-square statistic for a continuous distribution. Here, the equiprobable case can be used. Assuming that `data` contains observations coming from the uniform distribution, the [0, 1] interval is divided into 1/p subintervals, where p = `minExp`/n, n being the sample size, i.e., the number of observations stored in `data`. For each subinterval, the method counts the number of contained observations and the chi-square statistic is computed using `chi2Equal`. We usually choose `minExp` = 10.

  Parameters:
    `data` - array of observations in [0, 1)
    `minExp` - minimum number of expected observations in each subintervals
  Returns:
    the chi-square statistic for a continuous distribution

### chi2Equal

    public static double chi2Equal(DoubleArrayList data)

  Equivalent to `chi2Equal (data, 10)`.

  Parameters:
    `data` - array of observations in [0, 1)
  Returns:
    the chi-square statistic for a continuous distribution

### scan

    public static int scan(DoubleArrayList sortedData,
                           double d)

  Computes and returns the scan statistic SN(d ), defined in `FBar.scan`. Let U be the N observations contained into `sortedData`. The N observations in U[0..N - 1] must be real numbers in the interval [0, 1], sorted in increasing order. (See `FBar.scan` for the distribution function of SN(d )).

  Parameters:
    `sortedData` - sorted array of real-valued observations in the interval [0, 1]
    `d` - length of the test interval (∈(0, 1))
  Returns:
    the scan statistic

### cramerVonMises

    public static double cramerVonMises(DoubleArrayList sortedData)

  Computes and returns the Cramér-von Mises statistic WN2. It is defined by

  WN2 = 1/(12N) + ∑j=0N-1(U(j) - (j + 0.5)/N)2,

  assuming that `sortedData` contains U(0),..., U(N-1) sorted in increasing order.

  Parameters:
    `sortedData` - array of sorted real-valued observations in the interval [0, 1]
  Returns:
    the Cramér-von Mises statistic

### watsonG

    public static double watsonG(DoubleArrayList sortedData)

  Computes and returns the Watson statistic GN. It is defined by

  GN | = | (N)1/2max0 <= j <= N-1{(j + 1)/N - U(j) + bar(U)N -1/2} |
  | = | (N)1/2(DN+ + bar(U)N - 1/2), |

  where bar(U)N is the average of the observations U(j), assuming that `sortedData` contains the sorted U(0),..., U(N-1).

  Parameters:
    `sortedData` - array of sorted real-valued observations in the interval [0, 1]
  Returns:
    the Watson statistic GN

### watsonU

    public static double watsonU(DoubleArrayList sortedData)

  Computes and returns the Watson statistic UN2. It is defined by

  WN2 | = | 1/(12N) + ∑j=0N-1{U(j) - (j + 0.5)/N}2, |
  UN2 | = | WN2 - N(bar(U)N -1/2)2. |

  where bar(U)N is the average of the observations U(j), assuming that `sortedData` contains the sorted U(0),..., U(N-1).

  Parameters:
    `sortedData` - array of sorted real-valued observations in the interval [0, 1]
  Returns:
    the Watson statistic UN2

### andersonDarling

    public static double andersonDarling(DoubleArrayList sortedData)

  Computes and returns the Anderson-Darling statistic AN2. It is defined by

  AN2 | = | - N - 1/N ∑j=0N-1{(2j + 1)ln(U(j)) + (2N - 1 - 2j)ln(1 - U(j))}, |

  assuming that `sortedData` contains U(0),..., U(N-1).

  Parameters:
    `sortedData` - array of sorted real-valued observations in the interval [0, 1]
  Returns:
    the Anderson-Darling statistic

### kolmogorovSmirnov

    public static double[] kolmogorovSmirnov(DoubleArrayList sortedData)

  Computes the Kolmogorov-Smirnov (KS) test statistics DN+, DN-, and DN. It is defined by

  DN+ | = | max0 <= j <= N-1((j + 1)/N - U(j)), |
  DN- | = | max0 <= j <= N-1(U(j) - j/N), |
  DN | = | max (DN+, DN-). |

  and returns an array of length 3 that contains their values at positions 0, 1, and 2, respectively.

  These statistics compare the empirical distribution of U(1),..., U(N), which are assumed to be in `sortedData`, with the uniform distribution.

  Parameters:
    `sortedData` - array of sorted real-valued observations in the interval [0, 1]
  Returns:
    an array containing DN+, DN- and DN, in that order

### kolmogorovSmirnovJumpOne

    public static double[] kolmogorovSmirnovJumpOne(DoubleArrayList sortedData,
                                                    double a)

  Compute the KS statistics DN+(a) and DN-(a) defined in the description of the method `FDist.kolmogorovSmirnovPlusJumpOne`, assuming that F is the uniform distribution over [0, 1] and that U(1),..., U(N) are in `sortedData`. Returns an array of length 2 that contains their values at positions 0 and 1, respectively.

  Parameters:
    `sortedData` - array of sorted real-valued observations in the interval [0, 1]
    `a` - size of the jump
  Returns:
    an array containing DN+ and DN-, in that order

### pDisc

    public static double pDisc(double pL,
                               double pR)

  Computes a variant of the p-value p whenever a test statistic has a discrete probability distribution. This p-value is defined as follows:

  pL | = | P[Y <= y] |
  pR | = | P[Y >= y] |

  p = | pR, | if pR < pL,
  p = | 1 - pL, | if pR >= pL and pL < 0.5,
  p = | 0.5 | otherwise.

  The function takes pL and pR as input and returns p.

  Parameters:
    `pL` - left p-value
    `pR` - right p-value
  Returns:
    the p-value for a test on a discrete distribution

Overview | Package | Class | Tree | Deprecated | Index | Help
| SSJ V. 2.1.1.
PREV CLASS NEXT CLASS | FRAMES NO FRAMES
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD

To submit a bug or ask questions, send an e-mail to Pierre L'Ecuyer.