线性回归-多元线性回归
作者：互联网
| 符号 | 定义 |
|---|---|
| \(D\) | \(数据集,一个m\times (d+1)大小的矩阵X\) |
| \(m\) | \(样本量\) |
| \(d\) | \(维度,不含偏置项\) |
| \(X=\begin{pmatrix}x_{11} & x_{12} & ... & x_{1d} & 1 \\x_{21} & x_{22} & ... & x_{2d} & 1 \\...& ... & ... & ... & ... \\x_{m1} & x_{m2} & ... & x_{md} & 1 \\\end{pmatrix}=\begin{pmatrix}x_{1}^T & 1 \\x_{2}^T & 1 \\...& ... \\x_{m}^T & 1 \\\end{pmatrix}\) | \(样本矩阵\) |
| \(y=\begin{pmatrix}y_{1} \\y_{2} \\... \\y_{m} \\\end{pmatrix}\) | \(目标向量\) |
| \(\hat w = (w;b)\in R^{d+1}\) | \(回归系数向量\) |
1.多元线性回归模型
\(f(x_i)=w^Tx_i+b,使得f(x_i)\approx y_i\)
\(\hat w^*=argmin_{w}(y-X\hat w)^T(y-X\hat w)\)
2.多元线性回归的解析解
\(令E_{\hat w}=(y-X\hat w)^T(y-X\hat w)\)
\(求偏导 \frac{\partial E_{\hat w}}{\partial \hat w}=2X^T(X\hat w -y)\)
\(令\frac{\partial E_{\hat w}}{\partial \hat w} =0 即可求到最优解\)
\(当X^TX为满秩矩阵或者正定矩阵,可得\hat w^*=(X^TX)^{-1}X^Ty,令\hat x_i = (x_i,1),最最终回归模型为f(\hat x_i)=\hat x_i ^T (X^TX)^{-1}X^Ty\)
\(若不满秩,一般做法是引入正则化项\)
3.模型检验
\(F检验时对整体回归方程显著性的检验,即所有变量对被解释变量的显著性检验\)
\(构造统计量F\)
\(F=\frac{SSR/k}{SSE/(n-k-1)}=\frac{MSR}{MSE}\sim F(k,n-k-1)\)
标签：...,partial,回归,矩阵,多元,pmatrix,线性,frac,hat 来源： https://www.cnblogs.com/boyknight/p/15816248.html