Twisting properties

Starting with a sample \{x_1,\ldots,x_m\} observed from a random variable X having a given distribution law with a non-set parameter, a parametric inference problem consists of computing suitable values – call them estimates – of this parameter precisely on the basis of the sample. An estimate is suitable if replacing it with the unknown parameter does not cause major damage in next computations. In algorithmic inference, suitability of an estimate reads in terms of compatibility with the observed sample.

In turn, parameter compatibility is a probability measure that we derive from the probability distribution of the random variable to which the parameter refers. In this way we identify a random parameter Θ compatible with an observed sample. Given a sampling mechanism M_X=(g_\theta,Z), the rationale of this operation lies in using the Z seed distribution law to determine both the X distribution law for the given θ, and the Θ distribution law given an X sample. Hence, we may derive the latter distribution directly from the former if we are able to relate domains of the sample space to subsets of Θ support. In more abstract terms, we speak about twisting properties of samples with properties of parameters and identify the former with statistics that are suitable for this exchange, so denoting a well behavior w.r.t. the unknown parameters. The operational goal is to write the analytic expression of the cumulative distribution function F_\Theta(\theta), in light of the observed value s of a statistic S, as a function of the S distribution law when the X parameter is exactly θ.

Method

Given a sampling mechanism M_X=(g_\theta,Z) for the random variable X, we model \boldsymbol X=\{X_1,\ldots,X_m\} to be equal to \{g_\theta(Z_1),\ldots,g_\theta(Z_m)\}. Focusing on a relevant statistic S=h_1(X_1,\ldots,X_m) for the parameterθ, the master equation reads

s= h(g_\theta(z_1),\ldots, g_\theta(z_m))= \rho(\theta;z_1,\ldots,z_m).

When s is a well-behaved statistic w.r.t the parameter, we are sure that a monotone relation exists for each \boldsymbol z=\{z_1,\ldots,z_m\} between s and θ. We are also assured that Θ, as a function of \boldsymbol Z for given s, is a random variable since the master equation provides solutions that are feasible and independent of other (hidden) parameters.[1]

The direction of the monotony determines for any \boldsymbol z a relation between events of the type s\geq s'\leftrightarrow \theta\geq \theta' or vice versa s\geq s'\leftrightarrow \theta\leq \theta', where s' is computed by the master equation with \theta'. In the case that s assumes discrete values the first relation changes into s\geq s'\rightarrow \theta\geq \theta'\rightarrow s\geq s'+\ell where \ell>0 is the size of the s discretization grain, idem with the opposite monotony trend. Resuming these relations on all seeds, for s continuous we have either

F_{\Theta|S=s}(\theta)= F_{S|\Theta=\theta}(s)

or

F_{\Theta|S=s}(\theta)= 1-F_{S|\Theta=\theta}(s)

For s discrete we have an interval where F_{\Theta|S=s}(\theta) lies, because of \ell>0. The whole logical contrivance is called a twisting argument. A procedure implementing it is as follows.

Algorithm

Generating a parameter distribution law through a twisting argument
Given a sample \{x_1,\ldots,x_m\} from a random variable with parameter θ unknown,
  1. Identify a well behaving statistic S for the parameter θ and its discretization grain \ell (if any);
  2. decide the monotony versus;
  3. compute F_{\Theta}(\theta)\in\left(q_1(F_{S|\Theta=\theta}(s)),q_2(F_{S|\Theta=\theta}(s))\right) where:
    • if S is continuous q_1=q_2
    • if S is discrete
      1. q_2(F_S(s))=q_1(F_S(s-\ell) if s does not decrease with θ
      2. q_1(F_S(s))=q_2(F_S(s-\ell) if s does not increase with θ and
      3. q_i(F_S)= 1-F_S if s does not decrease with θ and q_i(F_S)= F_S if s does not increase with θ for i=1,2.

Remark

The rationale behind twisting arguments does not change when parameters are vectors, though some complication arises from the management of joint inequalities. Instead, the difficulty of dealing with a vector of parameters proved to be the Achilles heel of Fisher's approach to the fiducial distribution of parameters (Fisher 1935). Also Fraser’s constructive probabilities (Fraser 1966) devised for the same purpose do not treat this point completely.

Example

For \boldsymbol x drawn from a Gamma distribution, whose specification requires values for the parameters λ and k, a twisting argument may be stated by following the below procedure. Given the meaning of these parameters we know that

(k\leq k')\leftrightarrow(s_k \leq s_{k'}) for fixed λ, and
(\lambda\leq\lambda')\leftrightarrow(s_{\lambda'}\leq s_\lambda) for fixed k

where s_k=\prod_{i=1}^m x_i and s_\lambda=\sum_{i=1}^m x_i. This leads to a joint cumulative distribution function F_{\Lambda,K}(\lambda,k)=F_{\Lambda|k}(\lambda|k)F_K(k)=F_{K|\lambda}(k|\lambda)F_\Lambda(\lambda). Using the first factorization and replacing s_k with r_k=\frac{s_k}{s_\lambda^m} in order to have a distribution of K that is independent of \Lambda, we have

F_{\Lambda|k}(\lambda|k)=1 - \frac{\Gamma(k m, \lambda s_\Lambda)}{\Gamma(k m)}
F_K(k)=1-F_{R_k}(r_K)

with m denoting the sample size, s_{\Lambda} and r_K are the observed statistics (hence with indices denoted by capital letters), \Gamma(a,b) the Incomplete Gamma function and F_{R_k}(r_K) the Fox's H function that can be approximated with a Gamma distribution again with proper parameters (for instance estimated through the method of moments) as a function of k and m.

With a sample size m=30, s_\Lambda=72.82 and r_K= 4.5\times 10^{-46}, you may find the joint p.d.f. of the Gamma parameters K and \Lambda on the left. The marginal distribution of K is reported in the picture on the right.

Notes

  1. By default, capital letters (such as U, X) will denote random variables and small letters (u, x) their corresponding realizations.

References

This article is issued from Wikipedia - version of the Friday, July 25, 2014. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.