Eigenvalues and Diagonalization

Eigenvalues

When $Ax = \lambda x$ with $x \neq 0$, $\lambda$ is an eigenvalue with eigenvector $x$. Singular matrices have 0 as an eigenvalue.

The characteristic polynomial is

$$p_A(t) = \det(tI - A)$$

$\det(\lambda I - A) = 0 \Rightarrow \lambda$ is an eigenvalue, since $\lambda I - A$ has a nontrivial kernel. So the roots of $p_A$ are the eigenvalues.

$$\mathrm{eigspace}(\lambda) = \ker(\lambda I - A) = \mathrm{span}\{\text{eigenvectors of } \lambda\}$$

$\sigma(A)$ is the spectrum of $A$, the set of its eigenvalues.

• $\mathrm{almu}(\lambda)$ is the multiplicity of $\lambda$ as a root of $p_A$,
• $\mathrm{gemu}(\lambda) = \dim(\mathrm{eigspace}(\lambda))$,
• $1 \le \mathrm{gemu}(\lambda) \le \mathrm{almu}(\lambda)$.

The spectral radius is $\rho(A) = \max_{\lambda \in \sigma(A)} |\lambda|$.

Eigenvectors of Distinct Eigenvalues

If $\mathfrak{F}$ is a collection of eigenvectors of distinct eigenvalues, then $\mathfrak{F}$ is linearly independent.

Proof Suppose not, and take $x_1, \ldots, x_k \in \mathfrak{F}$ linearly dependent with $k$ least. Then

$$\alpha_1 x_1 + \cdots + \alpha_k x_k = 0$$

with all $\alpha_i \neq 0$ by minimality. Applying $A$,

$$\lambda_1 \alpha_1 x_1 + \cdots + \lambda_k \alpha_k x_k = 0$$

Subtracting $\lambda_1$ times the first equation,

$$(\lambda_2 - \lambda_1) \alpha_2 x_2 + \cdots + (\lambda_k - \lambda_1) \alpha_k x_k = 0$$

a dependence among $k - 1$ of them with nonzero coefficients, contradicting minimality.

Diagonalization

If $A, B$ are similar then $p_A = p_B$.

$A$ is diagonalizable when it's similar to a diagonal matrix. $A$ is diagonalizable $\iff$ $A$ has $n$ linearly independent eigenvectors.

Proof $A = SDS^{-1} \iff AS = SD$, which column by column reads

$$A[S^{(1)} | \cdots | S^{(n)}] = [S^{(1)} | \cdots | S^{(n)}] \begin{bmatrix} d_{11} & & \\ & \ddots & \\ & & d_{nn} \end{bmatrix}$$

i.e. $AS^{(i)} = d_{ii} S^{(i)}$, and the $S^{(i)}$ are linearly independent iff $S^{-1}$ exists.

So $n$ distinct eigenvalues $\Rightarrow$ diagonalizable.

Trace and Determinant

Let $S_i$ be the sum of all products of $i$ of the eigenvalues, and $E_i$ the sum of all $i \times i$ principal minors,

$$S_i = \sum_{1 \le j_1 < \cdots < j_i \le n} \lambda_{j_1} \cdots \lambda_{j_i}, \qquad E_i = \sum_{\substack{M \text{ an } i \times i \text{ principal} \\ \text{submatrix of } A}} \det M$$

So $S_1 = \sum \lambda_i$ and $E_1 = \mathrm{tr}(A)$. Expanding $p_A$ two ways,

$$\begin{aligned} p_A(t) &= t^n - S_1 t^{n-1} + S_2 t^{n-2} - \cdots + (-1)^n S_n \\ &= t^n - E_1 t^{n-1} + E_2 t^{n-2} - \cdots + (-1)^n E_n \end{aligned}$$

So $S_i = E_i$, which gives

$$\mathrm{tr}(A) = \sum_i \lambda_i, \qquad \det(A) = \prod_i \lambda_i$$

Block Triangular Matrices

$$\sigma\left(\begin{bmatrix} C_{11} & * \\ 0 & \ddots \end{bmatrix}\right) = \bigcup_i \sigma(C_{ii})$$

since

$$p_C(t) = \det(tI - C) = \det \begin{bmatrix} tI - C_{11} & * \\ 0 & \ddots \end{bmatrix} = \prod_i \det(tI - C_{ii})$$

$AB$ and $BA$

For $A \in M_{m,n}$, $B \in M_{n,m}$ with $n \ge m$,

$$p_{BA}(t) = t^{n-m} p_{AB}(t)$$

Proof

$$\begin{bmatrix} AB & 0 \\ B & 0 \end{bmatrix} \begin{bmatrix} I & A \\ 0 & I \end{bmatrix} = \begin{bmatrix} AB & ABA \\ B & BA \end{bmatrix} = \begin{bmatrix} I & A \\ 0 & I \end{bmatrix} \begin{bmatrix} 0 & 0 \\ B & BA \end{bmatrix}$$

so the two block matrices are similar, and their characteristic polynomials give $t^n p_{AB}(t) = t^m p_{BA}(t)$. In particular

$$\sigma(AB) \cup \{0\} = \sigma(BA) \cup \{0\}$$

Simultaneous Diagonalization

$A, B$ are simultaneously diagonalizable if there is an $S$ such that $S^{-1}AS$ and $S^{-1}BS$ are both diagonal.

For diagonalizable $A, B$: $AB = BA \iff A, B$ are simultaneously diagonalizable.

$(\Leftarrow)$ $A = SDS^{-1}$, $B = SD'S^{-1}$, so $AB = SDD'S^{-1} = SD'DS^{-1} = BA$.

If $A, B \in M_n$ commute, there is a permutation $\pi$ of $\{1, \ldots, n\}$ such that

$$\sigma(A + B) = \{\lambda_i(A) + \lambda_{\pi(i)}(B) : i = 1, \ldots, n\}, \qquad \sigma(AB) = \{\lambda_i(A) \lambda_{\pi(i)}(B) : i = 1, \ldots, n\}$$

Idempotent Matrices

Let $A^2 = A$. If $Ax = \lambda x$ then $\lambda x = A^2 x = \lambda^2 x$, so every eigenvalue is 1 or 0. Since $t^2 - t$ annihilates $A$ and has distinct roots, $A$ is diagonalizable (see minimal polynomial).

Then $A = SDS^{-1}$, $AS = SD$, where the first $r$ columns of $S$ are a basis for $\mathcal{E}_1$ and the rest a basis for $\mathcal{E}_0$,

$$A[\, S^{(1)} | \cdots | S^{(r)} \mid S^{(r+1)} | \cdots | S^{(n)} \,] = [\, S^{(1)} | \cdots | S^{(r)} \mid S^{(r+1)} | \cdots | S^{(n)} \,] D$$

For all $x \in \mathbb{C}^n$ there are unique $v \in \mathcal{E}_1$, $w \in \mathcal{E}_0$ with $x = v + w$, and $Ax = Av + Aw = v$. So $A$ is a projection onto $\mathcal{E}_1$ along $\mathcal{E}_0$.

Gersgorin Disks

For $A \in M_n$, the $i$th Gersgorin disk is centered at the diagonal entry with radius the moduli of the rest of the row summed,

$$D_i = \Big\{z \in \mathbb{C} : |z - a_{ii}| \le \sum_{j \neq i} |a_{ij}|\Big\}, \qquad G(A) = \bigcup_{i=1}^{n} D_i$$

$\sigma(A) \subseteq G(A)$, and if a connected component of $G(A)$ consists of $k$ disks, then exactly $k$ eigenvalues (counting multiplicity) are in it.