Introduction to Linear Algebra(7) Symmetric Matrices and Quadratic Forms

@[TOC](Introduction to Linear Algebra(7) Symmetric Matrices and Quadratic Forms)

Diagonolization Of Sysmmetric Matrices

If $A$A is symmetric, then any tow eigenvectors from different eigenspaces are orthogonal.
PROOF: $\lambda v_1 \cdot v_2 = (\lambda v_1)^{T}v_2=(Av_1)^{T}v_2=(v_1^{T}A^{T})v_2=v_1^T(Av_2)=\lambda_2 v_1\cdot v_2$λv1​⋅v2​=(λv1​)Tv2​=(Av1​)Tv2​=(v1T​AT)v2​=v1T​(Av2​)=λ2​v1​⋅v2​ for$\lambda_1 \ne \lambda_2$λ1​̸​=λ2​, $v_1 \cdot v_2 =0$v1​⋅v2​=0
An $n \times n$n×n matrix $A$A is orthogonally diagonalizable if and only if $A$A is a symmetric matrix.
The Spectral Theorem
An $n \times n$n×n sysmmetric matrix $A$A has the following properties:
a.A has $n$n real eigenvalues, counting multiplicities.
b. The demension of the eigenspace for each eigenvalues $lambda$lambda equals the multiplicity of $\lambda$λ as a root of the characterustic equation.
c. The eigenspaces are mutually orthogonal, in the sense that eigenvectors corresponding to different eigenvalues are orthogonal.
d. $A$A is orthogonally diagonalizable.
Spectral Decomposition
if $A$A is an sysmmetric matrix. then $A$A could be written as $A=\lambda_1u_1u_1^T+\lambda_2u_2u_2^T+\cdots+\lambda_nu_nu_n^T$A=λ1​u1​u1T​+λ2​u2​u2T​+⋯+λn​un​unT​

Quadratic Forms

The Principla Axes Theorem
Let $A$A be an $n \times n$n×n symmetric matrix. Then there is an orthogonal change of variable, $x=Py$x=Py, that transforms the quadratic form $x^TAx$xTAx into a quadratic form $y^TDy$yTDy with no cross-product term.

Classifying Quadratic Forms

A quadratic form $Q$Q is:
a.positive definite if $Q(x)>0$Q(x)>0 for all $X \ne 0$X̸​=0,
b. negative definite if $Q(x)&lt;0$Q(x)<0 for all $x \ne 0$x̸​=0,
c. indefinite if Q(x) assumes both positive and negative values.
Quadratic Forms and Eigenvalues
Let $A$A be an $n \times n$n×n symmetric matrix. Then a quadratic form $x^TAx$xTAx is:
a. positive definite if and only if the eigenvalues of $A$A are all positive,
b. negative definite if and only if the eigenvalues of $A$A are all negative,
c.indefinite if and only if $A$A has both positive and negative eigenvalues.
Let $A$A be a symmetric matrix, and define $m$m and $M$M . Then $M$M is the greatest eigenvalue $\lambda_1$λ1​ of $A$A amd $m$m is the least eigenvalue of $A$A. Then value of $x^TAx$xTAx is $M$M when $x$x is a unit eigenvector $u_1$u1​ corresponding to $M$M. Then value of $x^TAx$xTAx is $m$m when $x$x is a unit eigenvector corresponding to $m$m.
Let $A,\lambda_1$A,λ1​, and $u_1$u1​ be as in Theorem 6. Then the maximum value of $x^TAx$xTAx subject to the constraints $x^Tx=1,x^Tu_1=0$xTx=1,xTu1​=0
is the second greatset eigenvalue, $\lambda_2$λ2​, and this maximum is attained when $x$x is an eigenvector $u_2$u2​ corresponding to $\lambda_2$λ2​. This theorem is also extended to $\lambda_k$λk​

The Singular Value Decomposition

Singular Value Decomposition: a factorization $A=QDP^{-1}$A=QDP−1 is possible for any $m \times n$m×n matrix $A$A
Decomposition
Let $A$A be an $m \times n$m×n matrix with rank $r$r. Then there exists an $m \times n$m×n matrix $\Sigma$Σ as $\Sigma= \begin{bmatrix}D&amp;0\\ 0&amp;0 \end{bmatrix}$Σ=[D0​00​] for which the diagonal entries in $D$D are the first $r$r singular values of $A$A,$\sigma_1 \ge \sigma_2 \ge \cdots \ge \sigma_r &gt;0$σ1​≥σ2​≥⋯≥σr​>0, and there exist an $m \times m$m×m orthogonal matrix $U$U and an $n \times n$n×n orthogonal matrix $V$V such that $A=U \Sigma V^T$A=UΣVT
PROOF：
KaTeX parse error: Expected 'EOF', got '\lambd' at position 1: \̲l̲a̲m̲b̲d̲_i and $v_i$vi​ are the eigenvalues and eigenvectors of $A^TA$ATA seperately, so that $\{Av_1,\cdots,Av_r\}${Av1​,⋯,Avr​} is an orthogonal basis for $ColA$ColA. Normalize each $Av_i$Avi​ to obtain an orthonormal basis $\{u_1,\cdots,u_r\}${u1​,⋯,ur​}, where $u_i = \frac{Av_i}{||Av_i||}=\frac{Av_i}{\sigma_i}$ui​=∣∣Avi​∣∣Avi​​=σi​Avi​​ and $Av_i=\sigma u_i ,(\le i \le r)$Avi​=σui​,(≤i≤r)
Now extend $\{u_i,\cdots,u_r\}${ui​,⋯,ur​} to an orthonormal basis $\{u_1,\cdots,u_m\}${u1​,⋯,um​} of $R^m$Rm, and let$U=[u_1\quad u_2 \cdots \quad u_m] and V=[v_1\quad v_2 \cdots v_n]$U=[u1​u2​⋯um​]andV=[v1​v2​⋯vn​]
By construction, $U$U and $v$v are orthogonal matrices. $AV=[Av_1 \cdots Av_r \quad 0 \cdots 0 ]=[\sigma u_1 \cdots \sigma u_r \quad 0 \cdots 0 ]$AV=[Av1​⋯Avr​0⋯0]=[σu1​⋯σur​0⋯0]
Then$U\Sigma =AV$UΣ=AV Thus KaTeX parse error: Expected 'EOF', got '\SigmaV' at position 19: …\Sigma V^{-1}=U\̲S̲i̲g̲m̲a̲V̲^{T}
The Invertible Matrix Theorem (concluded)
Let $A$A be an $n \times n$n×n matrix. Then the following statements are each equivalent to the statement that $A$A is an invertible matrix.
u.$(Col A)^{\perp}={0}.$(ColA)⊥=0.
v.$(Nul A)^{\perp}=R^n.$(NulA)⊥=Rn.
w. $RowA=R^n.$RowA=Rn.
x. $A$A has $n$n nonzero sigular values.
Reduced $SVD$SVD and the Pseudoinverse of $A$A
let $r$r = rank$A$A then the $U$U and $V$V could be KaTeX parse error: Expected & or \\ or \cr or \end at end of input: …-r}^T=U_rDV_r^T
This factorization of $A$A is called a reduced singular value decomposition of $A$A. The following matrix is called the pseudoinverse of $A$A: $A^+=V_rD^{-1}U_r^T$A+=Vr​D−1UrT​

Principal Component Analysis

For simplicity, assume that the matrix $[X_1 \cdots X_N]$[X1​⋯XN​] is already in mean-deviation form. The goal of principal component analysis is to find an orthogonal $p \times p$p×p matrix $P=[u_1 \cdots u_p]$P=[u1​⋯up​] that determines a changeof variable, $X=PY$X=PY,or $\begin{bmatrix}x_1\\x_2\\ \vdots \\x_p \end{bmatrix}=\begin{bmatrix}u_1 &amp; u_2 &amp; \cdots &amp; u_p \end{bmatrix} \begin{bmatrix}y_1 \\y_2 \\ \vdots \\ y_p \end{bmatrix}$⎣⎢⎢⎢⎡​x1​x2​⋮xp​​⎦⎥⎥⎥⎤​=[u1​​u2​​⋯​up​​]⎣⎢⎢⎢⎡​y1​y2​⋮yp​​⎦⎥⎥⎥⎤​

标签：AAA,matrix,nn,Algebra,times,Forms,cdots,lambda,Linear
来源： https://blog.csdn.net/zbzhzhy/article/details/88218007