S-estimator

The goal of S-estimators is to have a simple high-breakdown regression estimator, which share the flexibility and nice asymptotic properties of M-estimators. The name "S-estimators" was chosen as they are based on estimators of scale.

We will consider estimators of scale defined by a function $\rho$, which satisfy

R1 - $\rho$ is symmetri, continuously differentiable and $\rho(0)=0$.

R2 - there exists $c > 0$ such that $\rho$ is strictly increasing on $[c, \infty[$

For any sample $\{r_1, ..., r_n\}$ of real numbers, we define the scale estimate $s(r_1, ..., r_n)$ as the solution of

${\textstyle \frac{1}{n}\sum_{i=1}^n \rho(r_i/s) = K}$,

where $K$ is the expectation value of $\rho$ for a standard normal distribution. (If there are more solutions to the above equation, then we take the one with the smallest solution for s; if there is no solution, then we put $s(r_1, ..., r_n)=0$ .)

Definition:

Let $(x_1, y_1), ..., (x_n, y_n)$ be a sample of regression data with p-dimensional $x_i$. For each vector $\theta$, we obtain residuals $s(r_1(\theta),..., r_n(\theta))$ by solving the equation of scale above, where $\rho$ satisfy R1 and R2. The S-estimator $\theta$ is defined by

$\text{minimize} \, s(r_1(\theta),..., r_n(\theta))$

and the final scale estimator is

$\theta = s(r_1(\theta), ..., r_n(\theta))$ .

^[1]

References

1. P. Rousseeuw and V. Yohai, Robust Regression by Means of S-estimators, from the book: Robust and nonlinear time series analysis, pages 256-272, 1984