Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
CS 130: Mathematical Methods in Computer Science Essentials of Linear Algebra Nestine Hope S. Hernandez Algorithms and Complexity Laboratory Department of Computer Science University of the Philippines, Diliman nshernandez@dcs.upd.edu.ph
Day 1
Motivating Problems
Systems of Linear Equations
Essentials of Linear Algebra
Motivating Problems Systems of Linear Equations Linear Systems in Matrix Form Solutions of Linear Systems
Solutions of Linear Systems
arrows in the circuit to represent theEquations direction of flow of the Solutions current;ofwhen Motivatinguse Problems Systems of Linear Linear Systems 10V + −
6Ω
I3
I3 I2
I2
a
b
2Ω I1
3Ω
4Ω
I1
+ −
5V Figure 1.2 A simple circuit with two loops, two
batteries, and four resistors.
arrows in the circuit to represent theEquations direction of flow of the Solutions current;ofwhen Motivatinguse Problems Systems of Linear Linear Systems 10V + −
6â„Ś
I3
I3 I2
I2
a
b
2â„Ś I1
3â„Ś
4â„Ś
I1
+ −
5V Figure 1.2 A simple circuit with two loops, two
and sum four resistors. Kircho 's currentbatteries, law: The of all current owing into a node equals the sum of the current owing out. Ohm's law: The voltage drop across a resistor is proportional to the current. Kircho 's voltage law: In any closed loop, the sum of the voltage drops must be zero.
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
A system of linear equations I1 7I1 −7I1
− I2 + 2I2 2I2
+
I3
+ 6I3 + 6I3
= 0 = 5 = 10 = 5
Motivating Problems 4
Systems of Linear Equations
Solutions of Linear Systems
A system of di erential equations
Essentials of linear algebra
A
B
Figure 1.1 Two tanks with inflows, outflows,
and connecting Let x1 be the amount of salt (inpipes. grams) in A at time t (in minutes) and let x2 be the amount of salt (in grams) in B at time t (in minutes).
dx1 x1 x2 = we 20 will − suppose+that the volume of solution in As a simplifying assumption, dt 20 80 each tank remains constant and all inflows and outflows happen at the identical rate of 5 liters per minute. that dx2 We will further assume x1 x2the tanks are uniformly = 35 in+each is identical − mixed so that the salt concentration dt 40 40 throughout the tank at a given time t . Let us now suppose that the volume of tank A is 200 liters; as we just noted, the pipe flowing into A delivers solution at a rate of 5 liters per minute. Moreover,
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Preliminaries x1 x1
+ +
2x2 x2
+ x3 + 2x3
= 1 = 0
equations.” For example, Motivating Problemsan “m × n system of Systems of Linear Equations x1 + 2x2 + x3 = 1 x1 + x2 + 2x3 = 0
Solutions of Linear Systems (1.2.2)
Preliminaries is a system of two linear equations in three unknown variables. A solution to the system is any point (x1 , x2 , x3 ) that makes both equations simultaneously true; the solution set for (1.2.2) is the collection of all such solutions. Geometrically, x1equations + describes 2x2 + = 1 3 three-dimensional each of these two a planexin space, as shown in figure 1.3, of all x1 and+hence the x2 solution + set 2xconsists = 0 points that lie on 3 both of the planes. Since the planes are not parallel, we expect this solution set to 10 x3
x1+ 2x2+ x3 = 1
5 x1+ x2+ 2x3 = 0 2 x1
2
x2
Figure 1.3 The intersection of the planes x1 + 2x2 +
x3 = 1 and x1 + x2 + 2x3 = 0.
equations.” For example, Motivating Problemsan “m × n system of Systems of Linear Equations x1 + 2x2 + x3 = 1 x1 + x2 + 2x3 = 0
Solutions of Linear Systems (1.2.2)
Preliminaries is a system of two linear equations in three unknown variables. A solution to the system is any point (x1 , x2 , x3 ) that makes both equations simultaneously true; the solution set for (1.2.2) is the collection of all such solutions. Geometrically, x1equations + describes 2x2 + = 1 3 three-dimensional each of these two a planexin space, as shown in figure 1.3, of all x1 and+hence the x2 solution + set 2xconsists = 0 points that lie on 3 both of the planes. Since the planes are not parallel, we expect this solution set to 10 x3
x1+ 2x2+ x3 = 1
5 x1+ x2+ 2x3 = 0 2 x1
1 1
2 1
1 2
1 0
2
x2
Figure 1.3 The intersection of the planes x1 + 2x2 +
x3 = 1 and x1 + x2 + 2x3 = 0.
equations.” For example, Motivating Problemsan “m × n system of Systems of Linear Equations x1 + 2x2 + x3 = 1 x1 + x2 + 2x3 = 0
Solutions of Linear Systems (1.2.2)
Preliminaries is a system of two linear equations in three unknown variables. A solution to the system is any point (x1 , x2 , x3 ) that makes both equations simultaneously true; the solution set for (1.2.2) is the collection of all such solutions. Geometrically, x1equations + describes 2x2 + = 1 3 three-dimensional each of these two a planexin space, as shown in figure 1.3, of all x1 and+hence the x2 solution + set 2xconsists = 0 points that lie on 3 both of the planes. Since the planes are not parallel, we expect this solution set to 10 x3
x1+ 2x2+ x3 = 1
5 x1+ x2+ 2x3 = 0 2 x1
1 1
2 1
1 2
1 0
2
x2
Figure 1.3 The intersection of the planes x1 + 2x2 +
x3 = 1 and x1 + x2 + 2x3 = 0.
→
Motivating Problems
De nition
Systems of Linear Equations
Solutions of Linear Systems
An m Ă— n matrix A is a rectangular array of mn real (or complex) numbers arranged in m horizontal rows and n vertical columns:
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
De nition
An m × n matrix A is a rectangular array of mn real (or complex) numbers arranged in m horizontal rows and n vertical columns:
a11 a21 : : : am1
a12 a22 : : : am2
··· ··· ··· ··· ··· ···
··· ··· ··· ··· ··· ···
··· ··· ··· aij ··· ···
a1n a2n : : : amn
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
De nition
An m × n matrix A is a rectangular array of mn real (or complex) numbers arranged in m horizontal rows and n vertical columns:
a11 a21 : : : am1
a12 a22 : : : am2
··· ··· ··· ··· ··· ···
··· ··· ··· ··· ··· ···
··· ··· ··· aij ··· ···
We often write the above matrix as A = [aij ].
a1n a2n : : : amn
Motivating Problems
Systems of Linear Equations
Examples A=
1 0
2 −1
3 5
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
Examples A=
1 0
2 −1
3 5
is a 2 × 3 matrix
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
Examples A=
1 0
1 B= 0 3
2 −1
3 5
2 5 −1
is a 2 × 3 matrix
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
Examples A=
1 0
1 B= 0 3
2 −1
3 5
is a 2 × 3 matrix
2 5 is a 3 × 2 matrix −1
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
Examples A=
1 0
1 B= 0 3 C=
1 −1
2 −1
3 5
is a 2 × 3 matrix
2 5 is a 3 × 2 matrix −1 2 5
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
Examples A=
1 0
1 B= 0 3 C=
1 −1
2 −1
3 5
is a 2 × 3 matrix
2 5 is a 3 × 2 matrix −1 2 5
is a 2 × 2 (square) matrix
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
Examples A=
1 0
2 −1
a=
1 −1 1
is a 2 × 3 matrix
2 5 is a 3 × 2 matrix −1
1 B= 0 3 C=
3 5
2
2 5
−1
is a 2 × 2 (square) matrix 5
0
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
Examples A=
1 0
2 −1
a=
1 −1 1
is a 2 × 3 matrix
2 5 is a 3 × 2 matrix −1
1 B= 0 3 C=
3 5
2
2 5
−1
is a 2 × 2 (square) matrix 5
0
is a 1 × 5 matrix
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
Examples A=
1 0
2 −1
a=
b=
1 −1 1
is a 2 × 3 matrix
2 5 is a 3 × 2 matrix −1
1 B= 0 3 C=
3 5
2 5
−1
2
1 2 −1 5 0
is a 2 × 2 (square) matrix 5
0
is a 1 × 5 matrix
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
Examples A=
1 0
2 −1
a=
b=
1 −1 1
is a 2 × 3 matrix
2 5 is a 3 × 2 matrix −1
1 B= 0 3 C=
3 5
2 5
−1
2
1 2 −1 5 0
is a 2 × 2 (square) matrix 5
0
is a 1 × 5 matrix
is a 5 × 1 matrix
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
Examples A=
1 0
2 −1
a=
b=
1 −1 1
is a 2 × 3 matrix
2 5 is a 3 × 2 matrix −1
1 B= 0 3 C=
3 5
2 5
−1
2
1 2 −1 5 0
is a 2 × 2 (square) matrix 5
0
is a 1 × 5 matrix
is a 5 × 1 matrix
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
More Examples a=
1
2
−1
5
0
and b =
1 2 −1 5 0
are vectors
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
More Examples a=
1
2
−1
5
0
and b =
1 D= 0 0
0 −1 0
0 0 4
1 2 −1 5 0
are vectors
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
More Examples a=
1
2
−1
5
0
and b =
1 D= 0 0
0 −1 0
1 2 −1 5 0
0 0 is a diagonal matrix 4
are vectors
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
More Examples a=
1
−1
2
5
0
and b =
0 −1 0
0 2 0 0
1 D= 0 0 2 0 E= 0 0
1 2 −1 5 0
0 0 is a diagonal matrix 4 0 0 2 0
0 0 0 2
are vectors
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
More Examples a=
1
−1
2
5
0
and b =
0 −1 0
0 2 0 0
1 D= 0 0 2 0 E= 0 0
1 2 −1 5 0
0 0 is a diagonal matrix 4 0 0 2 0
0 0 is a scalar matrix 0 2
are vectors
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
More Examples a=
1
−1
2
5
0
and b =
0 −1 0
0 2 0 0
1 D= 0 0 2 0 E= 0 0
1 2 −1 5 0
0 0 is a diagonal matrix 4 0 0 2 0
0 0 is a scalar matrix 0 2
are vectors
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Linear Systems in Matrix Form Consider the linear system of m equations in n unknowns, a11 x1 + a12 x2 + · · · + a1n xn = b1 a21 x1 + a22 x2 + · · · + a2n xn = b2 : : : : am1 x1 + am2 x2 + · · · + amn xn = bm
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Linear Systems in Matrix Form Consider the linear system of m equations in n unknowns, a11 x1 + a12 x2 + · · · + a1n xn = b1 a21 x1 + a22 x2 + · · · + a2n xn = b2 : : : : am1 x1 + am2 x2 + · · · + amn xn = bm
This linear system can be written in matrix form as Ax =b a11 a21 where A = : am1
a12 a22 : am2
··· ··· ···
a1n a2n , x = : amn
x1 x2 , b = : xn
b1 b2 : bm
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Linear Systems in Matrix Form Consider the linear system of m equations in n unknowns, a11 x1 + a12 x2 + · · · + a1n xn = b1 a21 x1 + a22 x2 + · · · + a2n xn = b2 : : : : am1 x1 + am2 x2 + · · · + amn xn = bm
This linear system can be written in matrix form as Ax =b a11 a21 where A = : am1
a12 a22 : am2
··· ··· ···
a1n a2n , x = : amn
x1 x2 , b = : xn
b1 b2 : bm
The matrix A is called the coe cient matrix of the linear system, and the matrix obtained by adjoining b to A, written as [A : b] is called the augmented matrix of the linear system.
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Some Matrix Operations Matrix Addition
If A = [aij ] and B = [bij ] are m × n matrices, then the sum of A and B is the m × n matrix C = [cij ], de ned by cij = aij + bij , 1 ≤ i ≤ m, 1 ≤ j ≤ n. That is, C is obtained by adding corresponding elements of A and B .
Matrix Multiplication
If A = [aij ] is an m × p matrix and B = [bij ] is a p × n matrix, then the product of A and B , denoted by AB , is the m × n matrix C = [cij ], de ned by cij = ai1 b1j + ai2 b2j + · · · + aip bpj = 1 ≤ i ≤ m, 1 ≤ j ≤ n.
Matrix Equality
p P
aik bkj
k=1
Two m × n matrices A = [aij ] and B = [bij ] are said to be equal if aij = bij , 1 ≤ i ≤ m, 1 ≤ j ≤ n, that is, if corresponding elements are equal.
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Elementary Row Operation De nition An elementary row operation on an m Ă— n matrix A = [aij ] is any of the following operations: (a) Interchange rows r and s of A. (b) Multiply row r of A by c 6= 0. (c) Add d times row r of A to row s of A, r 6= s.
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Elementary Row Operation De nition An elementary row operation on an m Ă— n matrix A = [aij ] is any of the following operations: (a) Interchange rows r and s of A. (b) Multiply row r of A by c 6= 0. (c) Add d times row r of A to row s of A, r 6= s.
Observe that when a matrix is viewed as the augmented matrix of a linear system, the elementary row operations are equivalent, respectively, to
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Elementary Row Operation De nition An elementary row operation on an m Ă— n matrix A = [aij ] is any of the following operations: (a) Interchange rows r and s of A. (b) Multiply row r of A by c 6= 0. (c) Add d times row r of A to row s of A, r 6= s.
Observe that when a matrix is viewed as the augmented matrix of a linear system, the elementary row operations are equivalent, respectively, to interchanging two equations,
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Elementary Row Operation De nition An elementary row operation on an m Ă— n matrix A = [aij ] is any of the following operations: (a) Interchange rows r and s of A. (b) Multiply row r of A by c 6= 0. (c) Add d times row r of A to row s of A, r 6= s.
Observe that when a matrix is viewed as the augmented matrix of a linear system, the elementary row operations are equivalent, respectively, to interchanging two equations, multiplying an equation by a nonzero constant,
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Elementary Row Operation De nition An elementary row operation on an m Ă— n matrix A = [aij ] is any of the following operations: (a) Interchange rows r and s of A. (b) Multiply row r of A by c 6= 0. (c) Add d times row r of A to row s of A, r 6= s.
Observe that when a matrix is viewed as the augmented matrix of a linear system, the elementary row operations are equivalent, respectively, to interchanging two equations, multiplying an equation by a nonzero constant, and adding a multiple of one equation to another equation.
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Elementary Row Operation De nition An elementary row operation on an m × n matrix A = [aij ] is any of the following operations: (a) Interchange rows r and s of A. (b) Multiply row r of A by c 6= 0. (c) Add d times row r of A to row s of A, r 6= s.
Observe that when a matrix is viewed as the augmented matrix of a linear system, the elementary row operations are equivalent, respectively, to interchanging two equations, multiplying an equation by a nonzero constant, and adding a multiple of one equation to another equation. An m × n matrix A is said to be row equivalent to an m × n matrix B if B can be obtained by applying a nite sequence of elementary row operations to A.
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
(Reduced) Row Echelon Form De nition
An m Ă— n matrix A is said to be in reduced row echelon form if it satis es the following properties: (a) All zero rows, if there are any, appear at the bottom of the matrix. (b) The rst nonzero entry from the left of a nonzero row is a 1.This entry is called a leading one of its row. (c) For each nonzero row, the leading one appears to the right and below any leading one's in preceding rows. (d) If a column contains a leading one, then all other entries in that column are zeros.
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
(Reduced) Row Echelon Form De nition
An m Ă— n matrix A is said to be in reduced row echelon form if it satis es the following properties: (a) All zero rows, if there are any, appear at the bottom of the matrix. (b) The rst nonzero entry from the left of a nonzero row is a 1.This entry is called a leading one of its row. (c) For each nonzero row, the leading one appears to the right and below any leading one's in preceding rows. (d) If a column contains a leading one, then all other entries in that column are zeros. An m Ă— n matrix satisfying properties (a), (b), and (c) is said to be in
row echelon form.
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Transform a Matrix to Row Echelon Form (REF)
0 2 0 0 2 2 2 0
3 −4 1 2 3 4 −5 2 4 −6 9 7
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Transform a Matrix to Reduced Row Echelon Form (RREF)
0 2 0 0 2 2 2 0
3 −4 1 2 3 4 −5 2 4 −6 9 7
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Gauss-Jordan reduction The Gauss-Jordan reduction procedure for solving the linear system Ax = b is as follows: Step 1. Form the augmented matrix [A : b]. Step 2. Transform [A : b] to reduced row echelon form [C : d] by using elementary row operations. Step 3. For each nonzero row of the matrix [C : d], solve the corresponding equation for the unknown associated with the leading one in that row.
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Gauss-Jordan reduction The Gauss-Jordan reduction procedure for solving the linear system Ax = b is as follows: Step 1. Form the augmented matrix [A : b]. Step 2. Transform [A : b] to reduced row echelon form [C : d] by using elementary row operations. Step 3. For each nonzero row of the matrix [C : d], solve the corresponding equation for the unknown associated with the leading one in that row.
Example x 2x 3x
+ + +
y 4y 6y
+ 2z − 3z − 5z
= = =
9 1 0
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Gaussian Elimination The Gaussian elimination procedure for solving the linear system Ax = b is as follows: Step 1. Form the augmented matrix [A : b]. Step 2. Transform [A : b] to (some) row echelon form [C : d] by using elementary row operations. Step 3. Solve the linear system corresponding to [C : d] by back substitution.
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Gaussian Elimination The Gaussian elimination procedure for solving the linear system Ax = b is as follows: Step 1. Form the augmented matrix [A : b]. Step 2. Transform [A : b] to (some) row echelon form [C : d] by using elementary row operations. Step 3. Solve the linear system corresponding to [C : d] by back substitution.
Example x 2x 3x
+ + +
y 4y 6y
+ 2z − 3z − 5z
= = =
9 1 0
Motivating Problems
Systems of Linear Equations
For Self-Study
Solutions of Linear Systems
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Properties of Matrix Addition Theorem Let A, B, C, and D be
m×n
matrices.
(a) A + B = B + A (b) A + (B + C) = (A + B) + C (c) There is a unique m × n matrix O
such that A + O = A m × n matrix A. The matrix O , all of whose entries are 0,is called the m × n additive identity or zero matrix. for any
(d)
m×n A + D = O. We shall write D as −A, so that A + (−A) = O . The matrix (−A) is called the additive inverse or For each matrix
D
m×n
matrix A, there is a unique
such that
negative of A.
Motivating Problems
Systems of Linear Equations
Solutions of Linear Systems
Properties of Matrix Multiplication Theorem (a)
If A, B, and C are of the appropriate sizes, then
A(BC) = (AB)C .
(b)
If A, B, and C are of the appropriate sizes, then
(c)
If A, B, and C are of the appropriate sizes, then
A(B + C) = AB + AC . (A + B)C = AC + BC .
(d)
n × n matrix In such that AIn = A m × n matrix A. Similarly, for any n × k matrix B , In B = B . 1 0 ··· 0 0 1 ··· 0 The n × n scalar matrix In = : : · · · : all of whose diagonal 0 0 ··· 1 entries are 1, is called the identity matrix of order n. There is a unique
for any
Motivating Problems
Systems of Linear Equations
Questions? See you next meeting!
Solutions of Linear Systems