What is the difference between "singular value" and "eigenvalue"?
I am trying to prove some statements about singular value decomposition, but I am not sure what the difference between singular value and eigenvalue is.
Is "singular value" just another name for eigenvalue?
$\endgroup$ 88 Answers
$\begingroup$The singular values of a $M\times N$ matrix $X$ are the square roots of the eigenvalues of the $N\times N$ matrix $X^*\,X$ (where $^*$ stands for the transpose-conjugate matrix if it has complex coefficients, or the transpose if it has real coefficients).
Thus, if $X$ is $N\times N$ real symmetric matrix with non-negative eigenvalues, then eigenvalues and singular values coincide, but it is not generally the case!
$\endgroup$ 5 $\begingroup$Given a matrix $A$, if the eigenvalues of $A^HA$ are $\lambda_i \geq 0$, then $\sqrt{\lambda_i}$ are the singular values of $A$. If $t$ is an eigenvalue of $A$, then $|t|$ is a singular value of $A$. And here is an example should be noticed, $$A = \begin{pmatrix}1&0&1\\0&1&1\\0&0&0\end{pmatrix},$$ the eigenvalues of $A$ are $1,1,0$ while the singular values of $A$ are $\sqrt{3},1,0$.
$\endgroup$ 6 $\begingroup$is singular value just another name for eigenvalue?
No, singular values & eigenvalues are different.
What is the difference between Singular Value and Eigenvalue?
There are many possible answers to this question. Since I don't know what you're trying to prove, I'd recommend carefully comparing definitions between the two: eigendecomposition, singular value decomposition
[EDIT: You might find the first several chapters of the book "Numerical Linear Algebra" by Trefethen and Bau more useful than the Wikipedia article. They're available here.]
Two important points:
Notice in particular that the SVD is defined for any matrix, while the eigendecomposition is defined only for square matrices (and more specifically, normal matrices).
Notice that singular values are always real, while eigenvalues need not be real.
A very clear explanation from Cleve Moler's text book
An eigenvalue and eigenvector of a square matrix A are a scalar λ and a nonzero vector x so that$$Ax = λx.$$A singular value and pair of singular vectors of a square or rectangular matrix A are a nonnegative scalar σ and two nonzero vectors u and v so that
$$Av = σu,$$$$A^Hu = σv.$$
Eigenvectors and singular vectors are them same if $A$ is a real symmetric matrix (so $A^H = A$).
$\endgroup$ $\begingroup$Singular values of the SVD decomposition of the matrix A is the square root of the eigenvalues of the matrix ($A$ multiplied by $A^T$) or ($A^T$ multiplied by $A$), the two are identical with positive eigenvalues.
$\endgroup$ $\begingroup$Consider the comparison between largest singular value and largest eigen value of a matrix with real entries, $A\in \mathbb{R}^{m \times n}$:
\begin{align*} \sigma_{\max}(A)&= \sup_{x \in S^{n-1}} ||Ax||_2 =\sup_{x \in S^{n-1}} \sqrt{||Ax||_2^2}= \sup_{x \in S^{n-1}} \sqrt{x^\top A^\top A x} = \sqrt{\lambda_{\max}(A^\top A)} \end{align*}
For $A^\top =A$, we have, $\sigma_{\max}(A)=\sqrt{\lambda_{\max}(A^2)}=\sqrt{\lambda^2_{\max}(A)}=|\lambda_{\max}(A)|=\sup\limits_{x \in S^{n-1}}| x^\top A x|$
$\endgroup$ $\begingroup$The right answer is well explained in THIS VIDEO .
$\endgroup$ $\begingroup$The eigenvalues $\mathbf{\Lambda}$ of the sample matrix $\mathbf{X}$ can be obtained applying SVD to the covariance of $\mathbf{X}$:
$$\mathbf{S} = cov(\mathbf{X}) \mapsto \mathbf{S}=\mathbf{U}\mathbf{\Lambda}\mathbf{V}^{T} $$
$\endgroup$
