Egwald Mathematics: Linear Algebra

Matrices

by

Elmer G. Wiens

Egwald's popular web pages are provided without cost to users.
Please show your support by joining Egwald Web Services as a Facebook Fan:
Follow Elmer Wiens on Twitter:

matrix definition | matrix operations | vector operations | matrix properties | special matrices | orthogonal matrices | inverse matrix
compute inverse | matrix determinant | eigenvalues and eigenvectors | eigenpair decomposition | jordan normal form | jordan decomposition
matrix decomposition | gauss decomposition | gram-schmidt decomposition | householder decomposition | jacobi decomposition
singular value decomposition | references

Definition of a Matrix.

A matrix A = [a_{i, j} ] is a rectangular m by n array of numbers (or other mathematical objects) expressed as:

such as:

A matrix of dimension m by 1 is called a column vector; a matrix of dimension 1 by n is called a row vector.

The Zero Matrix

The zero matrix O = [o_{i, j}], with o_{i, j} = 0 for all i and j.

The Identity Matrix.

A square matrix is an identity matrix I = [i_{i, j}], if  i_{i, j} = 1 where i = j, and i_{i, j} = 0 otherwise. Thus, an identity matrix is a diagonal matrix with 1's along its diagonal and zeros elsewhere. A 3 by 3 identity matrix appears below.

Rules of Operations on Matrices.

Scalar Multiplication of a Matrix.

The product of a matrix A by the scalar (real or complex number) s is given by:

or s * A = [s * a_{i, j}], with each entry multiplied by the number s.

Addition of Matrices.

Given matrices A = [a_{i, j}] and B = [b_{i, j} ] of dimension m by n, their sum A + B is another matrix C = [c_{i, j} ] of dimension m by n, where:

c_{i, j} = a_{i, j} + b_{i, j} , for i = 1, . . . m; j = 1, . . . n .

For example,

Matrix addition is commutative, since A + B = B + A.

Subtraction of Matrices.

Given matrices A and B of dimension m by n, their difference A - B is another matrix C of dimension m by n, where:

C = A - B = A + (-1) * B.

Equality of Matrices

Matrices A and B with the same dimensions are equal if A - B equals the zero matrix O.

Multiplication of two Matrices.

Given matrices A = [a_{i, j}] of dimension m by r and B = [b_{i, j} ] of dimension r by n, their product A * B is another matrix C = [c_{i, j} ] of dimension m by n, where:

c_{i, j} = a_{i, 1} * b_{1, j} + a_{i, 2} * b_{2, j} + . . . . . a_{r, 1} * b_{1, r} , for i = 1, . . . m; j = 1, . . . n

For example,

Two matrices are conformable for multiplication if their dimensions permit multiplication.

Two matrices are commutative under multiplication if A * B = B * A.

Product of a Matrix and a Vector.

The product of a column vector x = [x_i] and a matrix A = [a_{i, j}] is a column vector z, where :

z = A * x.

For example,

The product of a row vector y = [y_i] and a matrix A = [a_{i, j}] is a row vector w, where :

w = y * A.

For example,

The Transpose of a Matrix.

The transpose of an m by n matrix A = [a_{i, j}] is an n by m matrix A^T = [a_{j, i}] formed by interchanging the rows and columns of A.

The transpose of a column vector x is a row vector x^T, and vice versa.

From this point, vectors denoted by underlined small case letters, x, are column vectors, while row vectors have the superscript ^T, x^T.

Given matrices A = [a_{i, j}] of dimension m by r and B = [b_{i, j} ] of dimension r by n, the transpose of their product obeys:

(A * B)^T = B^T * A^T.

Vector Operations.

Vectors and their operations are described on the linear algebra: vectors web page.

Inner Product.

The inner product of a column vector x = [x_i] (n by 1) and a row vector y^T = [y_i] (1 by n) is their product as matrices:

y^T * x = y₁ * x₁ + y₂ * x₂ + . . . . + y_n * x_n

For example,

Outer Product.

The outer product of a column vector x = [x_i] (n by 1) and a row vector y^T = [y_i] (1 by n) is a matrix Z = [z_{i, j}] = x * y^T, where z_{i, j} = x_i * y_j.

For example,

Norm of a Vector.

The Euclidean norm ||x|| of a column vector x = [x_i] is the square root of  x^T * x, ie ||x|| = (x^T * x)^1/2.

For example, if v^T = (3, -6, 5), ||v|| = sqrt(9 + 36 + 25) = 8.3666.

The different types of vector norms and their properties are described on the vectors web page.

Properties of Matrices

A Square Matrix.

A square matrix has the same number of columns and rows.

A Symmetric Matrix.

A square matrix A is symmetric if A = A^T.

Given matrices A and A^Tdefined above, then the matrices:

are symmetric matrices.

A is skew-symmetric if A = -A^T.

A Triangular Matrix.

Diagonal Matrix.

A square matrix A is a diagonal matrix if all entries are zero, except possibly the entries on A's diagonal.

The Trace of a Matrix.

The trace of a square matrix A = [a_{i, j}] is tr(A) = a_{1, 1} + a_{2, 2} . . . . . + a_{n, n}, the sum of the diagonal elements of A.

The Rank of a Matrix.

The rank of an m by n matrix A, rank(A), equals the maximum number of A's linearly independent columns, which equals the maximum number of A's linearly independent rows. See the vectors web page for a definition of linearly dependent vectors.

The Quadratic Form of a Matrix.

Given a symmetric matrix A of dimension n, the function f(x) from Rⁿ to R is called a quadratic form if it is defined by:

f(x) = x^T * A * x,   for all vectors x in Rⁿ.

Positive Definite Matrix.

A real, symmetric matrix A is called a positive definite matrix if its quadratic form is positive for all nonzero x in Rⁿ. ie

f(x) = x^T * A * x > 0,   for all nonzero vectors x in Rⁿ.

Example.

Matrix Norms.

A matrix norm ||A|| of a square matrix A of dimension n is a measure of the "distance" of A from the zero matrix, O, with properties similar to those of a vector norm.

Induced (Natural) Matrix Norms.

Many important matrix norms are derived from vector norms. If  ||x||  is a vector norm for vectors of Rⁿ, the matrix norm induced by ||x|| is defined by:

||A||  =  max{ ||A * x||  |   all x in Rⁿ with ||x|| = 1 }

Properties of Matrix Norms.

For a given matrix norm || . ||, any scalar s, and square matrices A and B of dimension n:

The Frobenius Norm.

The Frobenius Norm, ||.||_F looks much like the Euclidean vector norm, but it is not the induced norm of the Euclidean norm.

For a given square matrix A of dimension n:

||A||_F = ( |a_1,1| + |a_1,2| + . . . . + |a_1,n|) + ( |a_2,1| + |a_2,2| + . . . . + |a_2,n|) + .   .   .   . + ( |a_n,1| + |a_n,2| + . . . . + |a_n,n|)

ie ||A||_F equals the sum of the absolute values of the entries of A.

Special Matrices

Permutation Matrix.

A permutation matrix P is formed by rearranging the columns of an identity matrix. The permutation matrix below will switch rows 1 and 3 of a matrix A if multiplied as P * A.

Rotation Matrices.

Rotation matrices play an important role in 3-D computer graphics. When a vector is multiplied by a rotation matrix, the vector is rotated through a given angle Ø.

The matrix Q_Ø with Ø = 0.524 radians is:

The next diagram shows the vectors   v and   Q_Ø * v  with  v = (5, 3) and an angle of rotation of  Ø = 0.524 radians.

You can change the vector, v, and the rotation angle, Ø in radians, in the form above to see how v and Q_Ø * v change. For example, try v = (-5, -3), Ø = 5 radians and click "Rotate Vector".

Properties of Rotation Matrices.

Projection Matrices.

The concept of projecting one vector onto another vector was used with vector dot products, and the Gram-Schmidt Process. When a vector is multiplied by the following matrix, the vector is projected onto the line that passes through the origin making an angle of Ø radians with the x-axis.

The matrix P_Ø with Ø = 0.524 radians is:

The next diagram shows the vectors   v and   P_Ø * v  with  v = (2, 4) and the projection line with an angle of   Ø = 0.524 radians.

You can change the vector, v, and the projection line angle, Ø in radians, in the form above to see how v and P_Ø * v change. For example, try v = (-4, 4), Ø = 5 radians and click "Project Vector".

Properties of Projection Matrices.

Reflection Matrices.

Reflection matrices act as a mirror with respect to the Ø line through the origin. When a vector is multiplied by a reflection matrix, the vector is reflected through the Ø line an equal distant to the other side of the line.

The matrix H_Ø with Ø = 0.785 radians is:

The next diagram shows the vectors   v and   H_Ø * v  with  v = (2, 4) and the reflection line with an angle of  Ø = 0.785 radians.

You can change the vector, v, and the reflection line angle, Ø in radians, in the form above to see how v and H_Ø * v change. For example, try v = (-4, 4), Ø = 2 radians and click "Reflect Vector".

Properties of Reflection Matrices.

Orthogonal and Orthonormal Matrices.

A square matrix A is orthogonal if for each column a_i of A,  a_i^T * a_j = 0 for any other column a_j of A.   If each column a_i of A has a norm of 1, then A is orthonormal. If A is orthonormal, then:

A^T * A = I, the identity matrix.

Example of an Orthonormal Matrix.

Permutation, rotation, and reflection matrices are orthonormal matrices.

Orthogonal / orthonormal matrices play an important role in the transformation and decomposition of matrices into special forms.

Inverse Matrix.

A square matrix A has an inverse if a matrix B exists with B * A = I.   A's inverse is usually written as A^-1.

Properties of A^-1.

If A is a square matrix of dimension n, then A^-1 obeys:

Example of a Matrix Inverse.

Computing the Inverse of a Matrix.

The Gauss-Jordan method permits one to calculate the inverse of a matrix. Begin by writing the matrix A and the identity matrix I side by side as shown in Tableau⁰ below.

Using the following row operations simultaneously on both matrices in the tableau, convert the left hand matrix to the identity matrix. When this is done, the right hand matrix is the inverse matrix.

Begin in the upper left hand corner of the left hand matrix. Proceed along the diagonal converting each column into the appropriate column of the identity matrix as shown in the sequence of tableaux. Only exchange rows if the diagonal entry is zero. Moreover, if the diagonal entry is zero, exchange rows with a lower row that has a nonzero entry in that column. If no suitable row exists, the matrix does not have an inverse.

Use the form below to compute the inverse of some other 3 by 3 A matrix.

Matrix Determinant.

The determinant of a square matrix A, det(A), is a scalar calculated as follows:

For example, compute det(A) by expanding by the 2nd row:

Properties of Determinants.

If A is square n by n matrix:

Eigenvalues and Eigenvectors.

The scalar and vector pair, (µ, v), is called an eigenvalue and eigenvector pair of the square matrix A if:

A *v = µ * v, or

(µ * I - A ) * v = 0.

Characteristic Equation.

The eigenvalues of A can be found by finding the roots of the polynomial in µ:

f(µ) = det(µ * I - A) = 0.

Computing Eigenpairs: Introduction.

For example, compute the eigenvalues and eigenvectors for the matrix:

Eigenvalues.

The characteristic equation is the polynomial:

The polynomial f(µ) has roots µ₁ = 4, and µ₂ = 2.

Eigenvectors.

Each eigenvalue µ_k provides a system of equations (µ_k * I - A ) * v = 0 that will yield the eigenvector(s) v_k associated with eigenvalue µ_k .

For the eigenvalue µ₁ = 4:

Equations e₁ and e₂ are the same, after dividing equation e₂ by 3. Solving for v₂ in terms of v₁ yields:

v₂ = v₁.

This means that the vector v₁^T = (1, 1), and any scalar multiple s * v₁, are eigenvectors associated with the eigenvalue µ₁ = 4.

The set of eigenvectors associated with the eigenvalue µ₁ = 4 is the kernel of the matrix A₁, ker(A₁), where A₁ = (4 * I - A ).

For the eigenvalue µ₂ = 2:

Equations e₁ and e₂ are the same. Solving for v₂ in terms of v₁ yields:

v₂ = 3 * v₁.

This means that the vector v₂^T = (1, 3), and any scalar multiple s * v₂, are eigenvectors associated with the eigenvalue µ₂ = 2.

The set of eigenvectors associated with the eigenvalue µ₂ = 2 is the kernel of the matrix A₂, ker(A₂), where A₂ = (2 * I - A ).

The next diagram shows    v₁,   A * v₁,   v₂,   A * v₂,   u = (1.5, -2),   and A * u.  See what happens if you set the vector u equal to a scalar multiple of one of the eigenvectors.

You can change the u vector in the form above to see how u and A * u change. For example, try u = (-1.5, 2) and click "Eigenvector".

Eigenpair Decomposition of A.

The matrix A has two unique eigenvalues and two linearly independent eigenvectors. These eigenvalues and eigenvectors can be used to decompose (factor) the matrix A as follows. Let S be the matrix whose first column is the eigenvector v₁, and whose second column is the eigenvector v₂. Let D be the diagonal matrix formed from the eigenvalues of A in the order of µ₁, µ₂.

By construction of S and D,  A * S  =  S * D. Since the columns of S are linearly independent, S has an inverse matrix, S^(-1). Multiplying A * S = S * D on the right by S^(-1) yields the eigenpairs decomposition of the matrix A as A   =   S * D * S^(-1).

The Determinant and Trace of the matrix A.

The determinant of A, det(A), equals the product of the eigenvalues of A.

det(A) = µ₁ * µ₂ = 4 * 2 = 8.

Consequently, the determinant of A is nonzero if and only if A has no zero eigenvalue. Furthermore, the matrix A is invertible if and only if A has no zero eigenvalue.

The trace of A, trace(A), equals the sum of the eigenvalues of A.

trace(A) = µ₁ + µ₂ = 4 + 2 = 6.

The general 2 by 2 matrix:

has trace(A) = a + d, and det(A) = a*d - b*c. Moreover, its characteristic equations is:

Factoring this quadratic equation yields the pair of eigenvalues, µ₁ and µ₂.

Jordan Normal Form.

A matrix with repeated eigenvalues has an eigenpair decomposition if it has a complete set of linearly independent eigenvectors. Otherwise, the Jordan Normal Form augmented with generalized eigenvectors is used to decompose the matrix.

For example, the matrix:

has the characteristic equation:

f(µ)   =   µ² - trace(A) * µ + det(A) =   µ² - 4 * µ + 4,

with eigenvalues µ₁ = 2, and µ₂ = 2. However, the only eigenvector is v^T₁ = (1, -1) and its multiples.

Recall that the eigenvector v^T₁ = (1, -1) is computed by solving the system of equations (2 * I - A ) * v = 0:

Equations e₁ and e₂ are the same. Solving for v₂ in terms of v₁ yields:

v₂ = -v₁.

This means that the vector v₁^T = (1, -1) (and its scalar multiples) is the only eigenvector associated with the eigenvalues µ_1,2 = 2.

Computing Generalized Eigenvectors: Introduction.

One can find a generalized eigenvalue v₂ for the eigenvalues µ_1,2 = 2 by solving the system of equations:

(A - 2 * I)*v₂ = v₁, or

(2 * I - A )*v₂ = - v₁.

Expanding:

Adding equation e₁ to equation e₂, and multiplying e₁ by -1, yields the echelon form:

The particular solution v^T_p = (1, 0) = v₂ is the generalized eigenvalue of the matrix A associated with the pair of equal eigenvalues µ_1,2 = 2,  while the homogeneous solution v_h is a multiple of the eigenvector v₁:

I obtained this result by using the echelon algorithm form on the linear equations web page to solve the system of equations.

As an eigenvector v₁ satisfies:

(A - 2 * I)*v₁ = o, or   A * v₁ = 2 * v₁.

As the generalized eigenvector constructed from v₁, v₂ satisfies:

(A - 2 * I)*v₂ = v₁, or   A * v₂ = 2 * v₂ + 1 * v₁.

Jordan Decomposition.

Let V be the matrix whose first column is the eigenvector v₁, and whose second column is the generaralized eigenvector v₂. Let J be the upper triangular matrix whose diagonal consists of eigenvalues of A,   µ_1,2 = 2,   and with a 1 in the upper right hand corner.

By construction of v₁ and v₂, A * V = V * J. Because v₁ and v₂ are linearly independent, the matrix V has an inverse. The Jordan Decomposition is:

Polynomials and their roots are described on the linear algebra: polynomials web page.

Systems of linear equations and their solutions are described on the linear algebra: linear equations web page.

Matrix Decompositions.

The eigenpair and Jordan decompositions of the matrices above are just two of the ways that matrices can be factored. Matrices can be decomposed into the product of other matrices, such as triangular, diagonal and/or orthogonal matrices. The decomposed matrices can be used to solve systems of linear equations.

The methods available to decompose a matrix depends on the dimensions of the matrix and on the rank of the matrix. A general matrix that is square and of full rank, ie its rows and columns are linearly independent, can be factored using Gaussian or Gram-Schmidt Decomposition. Matrices that are positive definite, symmetric, or triangular can be factored by more specific methods.

An m by n matrix with m > n can be factored with Householder or Jacobi transformations, or through singular value decomposition (svd).

An m by n matrix with m < n can be factored through singular value decomposition (svd).

Factor A Square Matrix of Full Rank

Gauss Matrix Decomposition.

The objective of Gaussian Decomposition is to write a matrix A as the product of a lower triangular matrix L and an upper triangular matrix U.

The Gaussian method is similar to the Gauss-Jordan method for computing a matrix's inverse. However, in each tableau the Gaussian method searches for the pivot with the largest absolute value and switches rows in both matrices accordingly. Moreover, it reduces the LHS matrix of the tableau to an upper triangular matrix U with the pivots along its diagonal, and the RHS matrix to a lower triangular matrix L with zeros along its diagonal, storing the multipliers used to zero out the U matrix below its diagonal. Furthermore, it keeps track of the row switches to construct a permutation matrix P. The actual L matrix is obtained from the RHS matrix of the final tableau by placing ones along its diagonal.

Gauss matrix decomposition yields matrices P, L, and U such that:

P * A = L * U, or

A = P^T * L * U.

Factor a matrix A so that: P * A = L *U.

Notice that in each tableau the relevant information in the L and U matrices fit onto one 5 by 5 matrix, which is what computer packages do to save storage space.

Compute the Determinant of A with Gauss Decomposition.

With the P * A = L * U decomposition the determinant of A, det(A), is the product of the diagonal elements of the matrix U, times +1 or -1 if an even or an odd number of row switches were used in the decomposition.

det(A) = 1 * (75) * (-2.6667) * (-0.65) * (-0.5385) * (-0.3571) = 25

Use the form below to factor some other square matrix A, up to 5 by 5.

If your matrix is smaller, specify its dimension and fill in / change the numbers in the upper left hand corner, and then click "Factor".

Gaussian Decomposition.

Gram-Schmidt Matrix Decomposition.

Given a square matrix A of dimension 4, the objective is to decompose A into two matrices Q and R, where Q is an orthonormal matrix, and R is an upper triangular matrix, such that:

A = Q * R.

Write A as consisting of 4 column vectors.

The Gram-Schmidt process is based on the method used on the vectors web page. The objective is to construct a set of 4 orthonormal vectors { q₁, q₂, q₃, q₄}, from the set of 4 linearly independent vectors { a₁, a₂, a₃, a₄}.

Begin by writing q₁ = a₁. Consider a₂. The projection of a₂ on q₁ is the vector:

p₁ = ((q₁ * a₂) / (q₁ * q₁)) * q₁.

The vector

q₂ = a₂ - p₁

is orthogonal to q₁, and {q₁, q₂} span the same vector subspace as {a₁, a₂}.

Repeat this process with the remaining { a₃, a₄} vectors. At stage k for a general matrix A, subtract the projections of a_k on the orthogonal vectors {q₁, ... q_k-1} from a_k to construct the next orthogonal vector q_k.

Do this at each stage k, by constructing new projection vectors, p_j, obtained by projecting a_k on each q_j as given by:

p_j = ((q_j * a_k) / (q_j * q_j)) * q_j,   for j = 1, . . . , k-1.

The required vector q_k orthogonal to {q₁, ... q_k-1} is:

q_k = a_k  -  p₁  -  p₂  -  .  .  .  -  p_k-1

The calculations for each step of the Gram-Schmidt decomposition for the example matrix are displayed below. The columns of the orthogonal matrix are divided by their norms to produce an orthonormal matrix.

Gram-Schmidt Decomposition A = Q * R

Orthonormal matrices have the property that Q^T * Q = I, the identity matrix. With A and Q known matrices, compute R as follows:

Q^T * A = Q^T * Q * R = R.

Expanding this last equation yields:

The entries below the diagonal of R are zero, because a_j is orthogonal to q_k for j < k.

The A = Q * R factorization for the example matrix is:

Use the form below to factor some other square matrix A, up to 5 by 5.

If your matrix is smaller, specify its dimension and fill in / change the numbers in the upper left hand corner, and then click "Factor".

Gram-Schmidt Decomposition.

Factor a Matrix with # Rows m >= # Columns n

Matrices with more rows than columns are found in least-squares problems. Three methods to decompose m by n matrices with m >= n are Householder, Jacobi, and singular value decomposition. Below I describe Householder and Jacobi transformations, leaving singular value decompositions for the case where m < n, ie where A has more columns than rows.

Householder Decomposition

Householder decomposition is used on the Statistics web pages to solve least-squares (multiple regression) problems.

Given an m by n matrix A, with rank(A) = n and with m >= n, factor A into the product of an orthonormal matrix Q and an upper triangular matrix R, such that:

A = Q * R.

The Householder decomposition is based on the fact that for any two different vectors, v and w, with ||v|| = ||w||, (ie different vectors of equal length), a reflection matrix H exists such that:

H * v = w.

To obtain the matrix H, define the vector u by:

u = (v - w) / (||v - w||).

The matrix H defined by:

H = I - 2 * u * u^T

is the required reflection matrix.

Observe that in the following 3-dimensional diagram,

To prove that H * v = w, observe that

Note that H = I - 2 * P, where P = u * u^T is the projection matrix onto the yellow line in the diagram above, along the vector (v + w).

Factor the following matrix as A = Q * R.

Given the matrix A, of dimension 4 by 3, the idea is to turn A into an upper triangular matrix using a sequence of 3 Householder transformations, H₁, H₂, H₃.

Starting with A₁ = A, let v₁ = the first column of A₁, and w₁^T = (norm(v₁), 0,...0), ie a column vector whose first component is the norm of v₁ with the remaining components equal to 0. The Householder transformation H₁ = I - 2 * u₁ * u₁^T with u₁ = v₁ - w₁ / ||v₁ - w₁|| will turn the first column of A₁ into w₁ as with H₁ * A1 = A₂. At each stage k, v_k = the kth column of A_k on and below the diagonal with all other components equal to 0, and w_k's kth component equals the norm of v_k with all other components equal to 0. Letting H_k * A_k = A_k+1, the components of the kth column of A_k+1 below the diagonal are each 0. These calculations are listed below for each stage for the matrix A.

A = Q * R

Use the form below to factor some other m by n matrix A, up to 5 by 4.

If your matrix is smaller, specify its dimension and fill in / change the numbers in the upper left hand corner, and then click "Factor".

Householder Decomposition.

Jacobi Decomposition

Jacobi (Givens Rotations) transformations are used to factor an m by n matrix A, with rank(A) = n and with m >= n, into the product of an orthonormal matrix Q and an upper triangular matrix R, such that:

A = Q * R.

Householder transformation are preferred to Jacobi transformations, because Householder transformations require fewer (almost one half) calculations.

A Jacobi transformation, J, is based on a rotation matrix, where the counter-clockwise angle of rotation, Ø, is selected to rotate a vector v^T = (v₁, v₂) into a vector w^T = (w₁, 0).

To get w₂ = 0 requires;

cos(Ø) = v₁ / ||v||,

sin(Ø) = -v₂ / ||v||, and

w₁ = norm(v) = ||v|| = sqrt(v₁*v₁ + v₂*v₂).

These values for the matrix J are computed directly from the vector v, without computing the angle Ø.

Starting in the upper left hand corner of a matrix A, such as:

zero out all entries below the diagonal of A in the first column with one Jacobi transformation per entry. Similarly, zero out the entries below the diagonal in the other columns of A.

Q * Q^T * A = A = Q * R

Use the form below to factor some other m by n matrix A, up to 5 by 4.

If your matrix is smaller, specify its dimension and fill in / change the numbers in the upper left hand corner, and then click "Factor".

Jacobi Decomposition.

Factor a Matrix with # Rows m <= # Columns n

Matrices with fewer rows than columns are found in under-determined problems, with fewer equations than unknown variables. Singular value decomposition is a convenient way to factor the matrix A where m < n, ie where A has more columns than rows.

However, any matrix can be factored into its singular value components. Furthermore, the matrices from the singular value decomposition can be combined to form the pseudoinverse (Moore-Penrose generalized inverse) of an arbitrary m by n matrix. If A is a square matrix of full rank, its pseudoinverse equals the matrix inverse. If A has independent columns, the pseudoinverse equals its left-inverse. If A has independent rows, its pseudoinverse equals its right-inverse.

Singular Value Decomposition

The Singular Value Decomposition (SVD) of a matrix A, m by n, is based on the following theorem of linear algebra:

Example of a Singular Value Decomposition.

A = U * S * V^T

The Pseudoinverse of A.

A⁺ = V * S⁺ * U^T

Use the form below to factor some other m by n matrix A, up to 4 by 5.

If your matrix is smaller, specify its dimension and fill in / change the numbers in the upper left hand corner, and then click "Factor".

Singular Value Decomposition.

Singular value decomposition works on a matrix of any dimensions, regardless of its rank.

References.

• Ayres, Frank Jr. Matrices. New York: Schaum McGraw-Hill, 1962.
• Ayres, Frank Jr. Modern Algebra. New York: Schaum McGraw-Hill 1965.
• Bretscher, Otto. Linear Algebra with Applications. Upper Saddle River: Prentice Hall, 1997.
• Burden, Richard L. and J. Douglas Faires. Numerical Analysis. 6th ed. Pacific Grove: Brooks/Cole, 1997.
• Campbell, Hugh D. Matrices with Applications. New York: Appleton 1968.
• Cohn, P. M. Linear Equations. London: Routledge, 1964.
• Demmel, James W. Applied Numerical Linear Algebra. Philadelphia: Siam, 1997.
• Dowling, Edward T. Mathematics for Economists. New York: Schaum McGraw-Hill, 1980.
• Goult, R. J., R. F. Hoskins, J.A. Milner, and M. J. Pratt. Computational Methods in Linear Algebra. New York: John Wiley, 1974
• Lipschutz, Seymour. Linear Algebra. New York, Schaum McGraw-Hill, 1968.
• Mathews, John H. and Kurtis D. Fink. Numerical Methods Using MATLAB. 3rd ed. Upper Saddle River: Prentice Hall, 1999.
• Press, William H., Brian P. Flannery, Saul A. Teukolsky, and William T. Vetterling. Numerical Recipes: The Art of Scientific Computing. Cambridge: Cambridge UP, 1989.
• Spiegel, Murray R. Vector Analysis and an Introduction to Tensor Analysis. New York: Schaum, 1959.
• Strang, Gilbert. Linear Algebra and Its Applications. 3d ed. San Diego: Harcourt, 1976.
• Varah, James. Numerical Linear Algebra: Computer Science 402 Lecture Notes. Vancouver: University of B.C., 2000.
• Watkins, David S. Fundamentals of Matrix Computations. New York: John Wiley, 1991.