MIT OpenCourseWare class notes

18.06 Linear Algebra, Lec 2: Matrix elimination method

Recorded lectures are on YouTube: MIT 18.06 Linear Algebra, Spring 2005

Lec 2: Matrix elimination method

In this lecture, we will introduce the elimination method, the most commonly used and fundamental approach for solving systems of linear equations Ax=b.

The elimination method involves performing appropriate operations on two equations within a system of equations to make the coefficient of a particular unknown variable zero, thereby reducing the number of unknowns.

Let's directly understand how to apply the elimination method through an example.

\(\begin{bmatrix} 1 & 2 & 1 \\ 3 & 8 & 1 \\ 0 & 4 & 1 \end{bmatrix} \begin{bmatrix} x \\ y\\ z \end{bmatrix} = \begin{bmatrix} 2 \\ 12 \\ 2 \end{bmatrix} \)

Perform simultaneous elimination on matrix A and vector b.

We refer to the 1 in the top left corner of the matrix as "the first pivot." The first step is to use elimination to change all the numbers in the first column, except for the pivot, to 0. The method is subtracting an appropriate multiple of the first row from each subsequent row; in this case, the second row should subtract three times the first row. After that, a similar operation should be performed on the third row. In this example, the numbers in the first column of the three rows are already 0, so no operation is needed.

Similarly, the 2 in the second row and second column (not 8, as it has already changed to 2 after the first elimination) is the second pivot. We need to use the same method to make all the numbers in the second column, except for the pivot and the number(s) above the pivot, equal to 0.

Continuing this way until the last row, we have completed eliminating A.

\( \begin{bmatrix} 1 & 2 & 1 \\ 3 & 8 & 1 \\ 0 & 4 & 1 \end{bmatrix} \quad \rightarrow \quad \begin{bmatrix} 1 & 2 & 1 \\ 0 & 2 & -2 \\ 0 & 4 & 1 \end{bmatrix} \quad \rightarrow \quad \begin{bmatrix} 1 & 2 & 1 \\ 0 & 2 & -2 \\ 0 & 0 & 5 \end{bmatrix} \\ \)

To synchronize with b, we have:

\(\begin{bmatrix} 2\\ 12\\ 2 \end{bmatrix} \quad \rightarrow \quad \begin{bmatrix} 2 \\ 6 \\ 2 \end{bmatrix} \quad \rightarrow \quad \begin{bmatrix} 2\\ 6 \\ -10 \end{bmatrix} \)

We found that all the elements in the lower half of the left side of matrix A after elimination are zero, so we call it the upper triangular matrix U. The result after elimination is denoted as c. We have transformed the original equation Ax = b into a new equation Ux = c.

From the last line, we get z = -2. By back-substituting, we can find y = 1 and x = 2.

The above method applies to any system of n equations with n unknowns.

In the example above, we performed elimination twice. Since the operations on A and b were done simultaneously, we might as well combine them into a single matrix for elimination; this new matrix is called the augmented matrix.

\( \begin{bmatrix} 1 & 2 & 1 & 2 \\ 3 & 8 & 1 & 12 \\ 0 & 4 & 1 & 2 \end{bmatrix} \quad \rightarrow \quad \begin{bmatrix} 1 & 2 & 1 & 2 \\ 0 & 2 & -2 & 6 \\ 0 & 4 & 1 & 2 \end{bmatrix} \quad \rightarrow \quad \begin{bmatrix} 1 & 2 & 1 & 2 \\ 0 & 2 & -2 & 6 \\ 0 & 0 & 5 & -10 \end{bmatrix} \)

Notice:

1) The pivot element cannot be zero. If elimination results in a row with a zero pivot position, a row exchange should be performed with the row below to ensure a non-zero number occupies the pivot position.

2) The elimination process terminates if a zero appears in the pivot position and there is no corresponding row below with a non-zero number; this means the system of linear equations has no unique solution.

Elimination Matrices

Next, we will discuss how to perform elimination using matrices. Before we do that, let me explain how to calculate the row. In the last lecture, we learned how to do column calculations (Ax). But what if the vector x is in front of A (xA)? In this situation, x cannot be written as a column vector but as a row vector, and xA equals a row vector. Here’s an example:

\(\begin{align*} &\begin{bmatrix} x & y & z \end{bmatrix} \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{bmatrix} = x \begin{bmatrix} 1 & 2 & 3 \end{bmatrix} +y \begin{bmatrix} 4 & 5 & 6 \end{bmatrix} +z \begin{bmatrix} 7 & 8 & 9 \end{bmatrix} \\ &=\begin{bmatrix} x+4y+7z & 2x+5y+8z & 3x+6y+9z \end{bmatrix} \end{align*}\)

Great! Now, let’s get to the topic of elimination matric!

Now, we want to use a matrix E21 to represent the first elimination operation on A. (The operation that eliminates the number on the second row, first column)

\( E_{21}\begin{bmatrix} 1 & 2 & 1 \\ 3 & 8 & 1 \\ 0 & 4 & 1 \end{bmatrix} =\begin{bmatrix} 1 & 2 & 1 \\ 0 & 2 & -2 \\ 0 & 4 & 1 \end{bmatrix}\)

The first row of E21 times A equals the first row of B(the matrix after the first elimination operation). According to the rules of multiplying a row vector by a matrix, we can determine that the first row of E21 is [1, 0, 0]. Similarly, the second row is [-3, 1, 0], and the third is [1, 0, 0].

\(E_{21}= \begin{bmatrix} 1 & 0 & 0 \\ -3 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix} \)

After the first operation, the number on the third row, in the first column, also becomes zero, which means the second operation is not needed. E31=I.

E32B=U. After calculation, we can get E32.

\(E_{32}= \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & -2 & 1 \end{bmatrix} \)

In sum:

\(E_{32}E_{21}A=EA=U\)

E is the product of all matrices involved in the elimination operations. Note that all elimination matrices must be written in order! For example, in this case, E = E32E21, not E21E32. This is because matrix multiplication is not commutative. The EA=U expression is a shorthand for eliminating A.

Since this article is too long, I will publish a short article to complete the remaining content of this lesson. Stay tuned!