Eigenvalues & Eigenvectors (NumPy)

Definition

Let $A$ be an $n\times n$ matrix (i.e. a square matrix). A non-zero vector $\overrightarrow{v}$ is an eigenvector of $A$ with eigenvalue $\lambda$ if

$$A\overrightarrow{v}=\lambda \overrightarrow{v}$$

Rewriting this equation, we see that $\overrightarrow{v}$ is a solution of the homogeneous system of equations

$$(A-\lambda I)\overrightarrow{v}=\overrightarrow{0}$$

where $I$ is the identity matrix of size $n$. Non-trivial solutions exists only when the matrix $A-\lambda I$ is noninvertible (singular). That is, when $\det (A-\lambda I)=0$. Therefore, the eigenvalues are the roots of the characteristic polynomial

$$p_A(\lambda )=\det (A-\lambda I)$$

Here are three examples that we will consider. In each case, we have pre-computed the eigenvalues and eigenvectors (check them yourself).

$$A=\left[\begin{array}{cc}2& 2\\ 1& 3\end{array}\right],{\lambda }_1=1,{\overrightarrow{v}}_1=\left[\begin{array}{c}-2\\ 1\end{array}\right],{\lambda }_2=4,{\overrightarrow{v}}_2=\left[\begin{array}{c}1\\ 1\end{array}\right]$$

$$B=\left[\begin{array}{cc}17& -15\\ 20& -18\end{array}\right],{\lambda }_1=2,{\overrightarrow{v}}_1=\left[\begin{array}{c}1\\ 1\end{array}\right],{\lambda }_2=-3,{\overrightarrow{v}}_2=\left[\begin{array}{c}3\\ 4\end{array}\right]$$

$$C=\left[\begin{array}{ccc}-3& 5& -5\\ -7& 9& -5\\ -7& 7& -3\end{array}\right],{\lambda }_1=-3,{\overrightarrow{v}}_1=\left[\begin{array}{c}1\\ 1\\ 1\end{array}\right],{\lambda }_2=4,{\overrightarrow{v}}_2=\left[\begin{array}{c}0\\ 1\\ 1\end{array}\right],{\lambda }_3=2,{\overrightarrow{v}}_3=\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right]$$

$$D=\left[\begin{array}{ccc}1& 2& -2\\ 2& 1& 2\\ -2& 2& 1\end{array}\right],{\lambda }_1=3,{\overrightarrow{v}}_{1,1}=\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right],{\overrightarrow{v}}_{1,2}=\left[\begin{array}{c}-1\\ 0\\ 1\end{array}\right],{\lambda }_2=-3,{\overrightarrow{v}}_2=\left[\begin{array}{c}1\\ -1\\ 1\end{array}\right]$$

Notice, for matrix $D$ there is one eigenvalue that has two associated eigenvectors.

Warning

Eigenvectors are not unique, any scalar multiple of an eigenvector is an eigenvector. Therefore, some caution is needed when having SciPy compute eigenvectors. The values returned will be in floating point form, if you expected an eigenvector with integer or rational number entries you may need to rescale it to get it into this form. We'll see examples of this below.

scipy.linalg.eig

import numpy as np
import scipy.linalg as la

The function scipy.linalg.eig computes eigenvalues and eigenvectors of a square matrix $A$. It returns a 2-tuple, the first part is a 1D array of eigenvalues, the second part is a 2D array where the columns are the (normalized) eigenvectors associated to each eigenvalue. The eigenvalues are returned as complex numbers, so if you expect real eigenvalues then check to see if the imaginary part is $0$. This will be the case in all our examples below.

Example

Determine the eigenvalues and eigenvectors of

$$A=\left[\begin{array}{cc}2& 2\\ 1& 3\end{array}\right]$$

A = np.array([[2,2],[1,3]])
la.eig(A)

(array([1.+0.j, 4.+0.j]),
 array([[-0.89442719, -0.70710678],
        [ 0.4472136 , -0.70710678]]))

The eigenvalues are $1$ and $4$, and the corresponding eigenvalues are $\left[\begin{array}{c}-0.89442719\\ 0.4472136\end{array}\right]$ and $\left[\begin{array}{c}-0.70710678\\ -0.70710678\end{array}\right]$. Notice, in the first vector it looks like the first component is twice the second component. Whereas in the second vector both components are equal. Therefore, these vectors are scalar multiples of $\left[\begin{array}{c}-2\\ 1\end{array}\right]$ and $\left[\begin{array}{c}1\\ 1\end{array}\right]$.

To check this directly in python we can use the tuple assignment to capture the arrays for eigenvalues and eigenvectors, and then pick off the two vectors.

evals, evecs = la.eig(A)
v1 = evecs[:,0].reshape(2,1)
v2 = evecs[:,1].reshape(2,1)
print(v1)
print(v2)

[[-0.89442719]
 [ 0.4472136 ]]
[[-0.70710678]
 [-0.70710678]]

v1/v1[1]

array([[-2.],
       [ 1.]])

Example

Determine the eigenvalues and eigenvectors of

$$B=\left[\begin{array}{cc}17& -15\\ 20& -18\end{array}\right]$$

B = np.array([[17,-15],[20,-18]])
la.eig(B)

(array([ 2.+0.j, -3.+0.j]),
 array([[0.70710678, 0.6       ],
        [0.70710678, 0.8       ]]))

The eigenvalues are $2$ and $-3$, and the corresponding eigenvectors are $\left[\begin{array}{c}0.70710678\\ 0.70710678\end{array}\right]$ and $\left[\begin{array}{c}0.6\\ 0.8\end{array}\right]$ which can be rescaled to $\left[\begin{array}{c}1\\ 1\end{array}\right]$ and $\left[\begin{array}{c}3\\ 4\end{array}\right]$.

Example

Determine the eigenvalues and eigenvectors of

$$C=\left[\begin{array}{ccc}-3& 5& -5\\ -7& 9& -5\\ -7& 7& -3\end{array}\right]$$

C= np.array([[-3,5,-5],[-7,9,-5],[-7,7,-3]])
la.eig(C)

(array([-3.+0.j,  4.+0.j,  2.+0.j]),
 array([[ 5.77350269e-01, -8.33283831e-16, -7.07106781e-01],
        [ 5.77350269e-01, -7.07106781e-01, -7.07106781e-01],
        [ 5.77350269e-01, -7.07106781e-01,  3.33066907e-16]]))

The eigenvalues are $-3$, $4$ and $2$, and the corresponding eigenvectors are $\left[\begin{array}{c}5.77350269e-01\\ 5.77350269e-01\\ 5.77350269e-01\end{array}\right]$, $\left[\begin{array}{c}-8.33283831e-16\\ -7.07106781e-01\\ -7.07106781e-01\end{array}\right]$, and $\left[\begin{array}{c}-7.07106781e-01\\ -7.07106781e-01\\ 3.33066907e-16\end{array}\right]$. The entry $3.33066907e-16$ is very very small compared to the other entries (e-16 means $\cdot {10}^{-16}$), which leads us to believe it should be $0$. Therefore, these vectors can be rescaled to $\left[\begin{array}{c}1\\ 1\\ 1\end{array}\right]$, $\left[\begin{array}{c}0\\ 1\\ 1\end{array}\right]$, and $\left[\begin{array}{c}1\\ 1\\ 0\end{array}\right]$.

Example

Determine the eigenvalues and eigenvectors of

$$D=\left[\begin{array}{ccc}1& 2& -2\\ 2& 1& 2\\ -2& 2& 1\end{array}\right]$$

D = np.array([[1,2,-2],[2,1,2],[-2,2,1]])
evals, evecs = la.eig(D)
print(evals)
print(evecs)

[ 3.+0.j -3.+0.j  3.+0.j]
[[ 0.81649658  0.57735027  0.22645541]
 [ 0.40824829 -0.57735027  0.79259392]
 [-0.40824829  0.57735027  0.56613852]]

The eigenvalue $3$ appears twice, therefore the first and third vectors both correspond to eigenvlaue $3$.

Eignevalue $3$ has eigenvectors $\left[\begin{array}{c}0.81649658\\ 0.40824829\\ -0.40824829\end{array}\right]$ and $\left[\begin{array}{c}0.22645541\\ 0.79259392\\ 0.56613852\end{array}\right]$.

Eignevalue $-3$ has eigenvector $\left[\begin{array}{c}0.57735027\\ -0.57735027\\ 0.57735027\end{array}\right]$, which rescales to $\left[\begin{array}{c}1\\ -1\\ 1\end{array}\right]$.

To rescale the two eigenvectors corresponding to eigenvalue $3$ we do the following.

v11 = evecs[:,0] # (1)
v11 = v11/v11[1] # (2)
v11

1. select the first column of the eigenvectors array.
2. rescale by dividing through by the second entry.

array([ 2.,  1., -1.])

v12 = evecs[:,2] # (1)
v12 = 2*v12/v12[0] # (2)
v12

1. select the third column of the eigenvectors array.
2. rescale by dividing through by the first entry (since it is the smallest). Once we do this we'll see that the numbers are fractions ending in $0.5$, so we scale by $2$.

array([2., 7., 5.])

Notice that these two eigenvectors are not orthogonal. One way to fix this is to use .eigh instead of eig since the matrix $A$ is symmetric.

D = np.array([[1,2,-2],[2,1,2],[-2,2,1]])
evals, evecs = la.eigh(D)
print(evals)
print(evecs)

[-3.  3.  3.]
[[ 0.57735027  0.          0.81649658]
 [-0.57735027  0.70710678  0.40824829]
 [ 0.57735027  0.70710678 -0.40824829]]    

Rescaling the vectors corresponding to eigenvalue $3$ we get $\left[\begin{array}{c}0\\ 1\\ 1\end{array}\right]$ and $\left[\begin{array}{c}2\\ 1\\ -1\end{array}\right]$.