最小二乘回归受数据中的离群点的影响较大,稳健回归通过降低离群点的影响缓解此问题。M估计法是稳健回归的重要方法之一,M 估计法的目标函数为:
m i n ∑ ρ ( ϵ i ) = m i n ∑ ρ ( y i − β ^ ∗ X i ) min\sum\rho(\epsilon_i) = min\sum\rho(y_i - \hat{\beta} * X_i) min∑ρ(ϵi)=min∑ρ(yi−β^∗Xi)
函数 ρ \rho ρ 具有特性:
- ρ ( ϵ ) ≥ 0 , ρ ( 0 ) = 0 \rho(\epsilon)\ge 0, \rho(0)=0 ρ(ϵ)≥0,ρ(0)=0
- ρ ( ϵ ) = ρ ( − ϵ ) \rho(\epsilon) = \rho(-\epsilon) ρ(ϵ)=ρ(−ϵ)
目标函数关于带估计参数 β ^ \hat{\beta} β^ 求导:
∑ ρ ( y i − β ^ X i ) ∂ β ^ = ∑ − ρ ( y i − β ^ X i ) ∂ β ^ X i ≜ ∑ ψ ( ρ ( y i − β ^ X i ) ) X i \frac{\sum\rho(y_i-\hat{\beta}X_i)}{\partial \hat{\beta}} = \sum -\frac{\rho(y_i - \hat{\beta}X_i)}{\partial \hat{\beta}}X_i\triangleq \sum \psi(\rho(y_i - \hat{\beta}X_i)) X_i ∂β^∑ρ(yi−β^Xi)=∑−∂β^ρ(yi−β^Xi)Xi≜∑ψ(ρ(yi−β^Xi))Xi
其中 ψ ( ϵ ) = ∂ ρ ( ϵ ) ∂ β ^ \psi(\epsilon) = \frac{\partial \rho(\epsilon)}{\partial \hat{\beta}} ψ(ϵ)=∂β^∂ρ(ϵ)
Andrews 估计
Andrews 1974年提出:
ψ ( z ) = { s i n ( z / c ) ∣ z ∣ ≤ c 0 ∣ z ∣ ≥ c \psi(z)=\left\{ \begin{aligned} sin(z/c) & & |z| \le c \\ 0 & & |z| \ge c \end{aligned} \right. ψ(z)={sin(z/c)0∣z∣≤c∣z∣≥c
ρ ( z ) = { 1 − c o s ( z ) ∣ z ∣ < π 0 ∣ z ∣ ≥ π \rho(z)=\left\{ \begin{aligned} 1 - cos(z) & & |z| < \pi \\ 0 & & |z| \ge \pi \end{aligned} \right. ρ(z)={1−cos(z)0∣z∣<π∣z∣≥π
Huber 估计
ψ ( z ) = { z ∣ z ∣ < c c ⋅ s i g n ( z ) ∣ z ∣ ≥ c \psi(z)=\left\{ \begin{aligned} z & & |z| < c \\ c\cdot sign(z) & & |z| \ge c \end{aligned} \right. ψ(z)={zc⋅sign(z)∣z∣<c∣z∣≥c
ρ ( z ) = { 1 2 z 2 ∣ z ∣ < c c ∣ z ∣ − 1 2 c 2 ∣ z ∣ ≥ c \rho(z)=\left\{ \begin{aligned} \frac 12 z^2 & & |z| < c \\ c|z| - \frac12c^2 & & |z| \ge c \end{aligned} \right. ρ(z)=⎩ ⎨ ⎧21z2c∣z∣−21c2∣z∣<c∣z∣≥c
Tuerky 估计
ψ ( z ) = { z ( c 2 − z 2 ) 2 ∣ z ∣ < c c ⋅ s i g n ( z ) ∣ z ∣ ≥ c \psi(z)=\left\{ \begin{aligned} z(c^2 - z^2)^2 & & |z| < c \\ c\cdot sign(z) & & |z| \ge c \end{aligned} \right. ψ(z)={z(c2−z2)2c⋅sign(z)∣z∣<c∣z∣≥c
ρ ( z ) = { 1 6 ( c 6 − ( c 2 − z 2 ) 3 ) ∣ z ∣ < c 0 ∣ z ∣ ≥ c \rho(z)=\left\{ \begin{aligned} \frac 16(c^6 - (c^2-z^2)^3) & & |z| < c \\ 0 & & |z| \ge c \end{aligned} \right. ρ(z)=⎩ ⎨ ⎧61(c6−(c2−z2)3)0∣z∣<c∣z∣≥c
L p L_p Lp 估计
ρ ( z ) = ∣ z ∣ p \rho(z) = |z|^p ρ(z)=∣z∣p