Cs130 day05

Subspaces of lRn

Vector Spaces

CS 130: Mathematical Methods in Computer Science Vector Spaces and Linear Systems Nestine Hope S. Hernandez Algorithms and Complexity Laboratory Department of Computer Science University of the Philippines, Diliman nshernandez@dcs.upd.edu.ph Day 5

Subspaces of lRn

Vector Spaces

Vector Spaces and Linear Systems

Vector Spaces Basis and Dimension

Subspaces of lRn Row(Column) Space of a Matrix Orthonormal Bases in lRn

Subspaces of lRn

Vector Spaces

Vector Spaces and Linear Systems

Vector Spaces Basis and Dimension

Subspaces of lRn Row(Column) Space of a Matrix Orthonormal Bases in lRn

Subspaces of lRn

Vector Spaces

Basis

The vectors v~1 , v~2 , · · · , v~k in a vector space V are said to form a basis for V if (a) v~1 , v~2 , · · · , v~k span V and (b) v~1 , v~2 , · · · , v~k are linearly independent.

Subspaces of lRn

Vector Spaces

Basis

The vectors v~1 , v~2 , · · · , v~k in a vector space V are said to form a basis for V if (a) v~1 , v~2 , · · · , v~k span V and (b) v~1 , v~2 , · · · , v~k are linearly independent.

Example Show that S = {t2 + 1, t − 1, 2t + 2} is a basis for the vector space P2 .

Subspaces of lRn

Vector Spaces

Basis

The vectors v~1 , v~2 , · · · , v~k in a vector space V are said to form a basis for V if (a) v~1 , v~2 , · · · , v~k span V and (b) v~1 , v~2 , · · · , v~k are linearly independent.

Example Show that S = {t2 + 1, t − 1, 2t + 2} is a basis for the vector space P2 .

If S = {v~1 , v~2 , · · · , v~k } is a basis for a vector space V , then every vector in V can be written in one and only one way as a linear combination of the vectors in S.

Subspaces of lRn

Vector Spaces

Basis

The vectors v~1 , v~2 , · · · , v~k in a vector space V are said to form a basis for V if (a) v~1 , v~2 , · · · , v~k span V and (b) v~1 , v~2 , · · · , v~k are linearly independent.

Example Show that S = {t2 + 1, t − 1, 2t + 2} is a basis for the vector space P2 .

If S = {v~1 , v~2 , · · · , v~k } is a basis for a vector space V , then every vector in V can be written in one and only one way as a linear combination of the vectors in S. Find a basis for the subspace W of P2 , consisting of all vectors of the form at2 + bt + c, where c = a − b.

Subspaces of lRn

Vector Spaces

Basis

The vectors v~1 , v~2 , · · · , v~k in a vector space V are said to form a basis for V if (a) v~1 , v~2 , · · · , v~k span V and (b) v~1 , v~2 , · · · , v~k are linearly independent.

Example Show that S = {t2 + 1, t − 1, 2t + 2} is a basis for the vector space P2 .

If S = {v~1 , v~2 , · · · , v~k } is a basis for a vector space V , then every vector in V can be written in one and only one way as a linear combination of the vectors in S. Find a basis for the subspace W of P2 , consisting of all vectors of the form at2 + bt + c, where c = a − b.

Let S = {v~1 , v~2 , · · · , v~k } be a set of nonzero vectors in a vector space V and let W = spanS. Then some subset of S is a basis for W .

Subspaces of lRn

Vector Spaces

Finite-Dimensional Vector Space

The dimension of a nonzero vector space V is the number of vectors in a basis for V .

Subspaces of lRn

Vector Spaces

Finite-Dimensional Vector Space

The dimension of a nonzero vector space V is the number of vectors in a basis for V .

If S is a linearly independent set of vectors in a finite-dimensional vector space V , then there is a basis T for V , which contains S.

Subspaces of lRn

Vector Spaces

Finite-Dimensional Vector Space

The dimension of a nonzero vector space V is the number of vectors in a basis for V .

If S is a linearly independent set of vectors in a finite-dimensional vector space V , then there is a basis T for V , which contains S.

Example Find a basis for lR4 that contains the vectors v1 = (1, 0, 1, 2), v2 = (0, 1, 1, 2) and v3 = (1, 1, 1, 3).

Subspaces of lRn

Vector Spaces

Finite-Dimensional Vector Space

Let V be an n-dimensional vector space, and let S = {v~1 , v~2 , 路 路 路 , v~n } be a set of n vectors in V . (a) If S is linearly independent, then it is a basis for V . (b) If S spans V , then it is a basis for V .

Subspaces of lRn

Vector Spaces

Finite-Dimensional Vector Space

Let V be an n-dimensional vector space, and let S = {v~1 , v~2 , 路 路 路 , v~n } be a set of n vectors in V . (a) If S is linearly independent, then it is a basis for V . (b) If S spans V , then it is a basis for V .

Exercise P3 is a 4-dimensional vector space.

Show that t3 + t2 + t + 1, t3 + 2t2 + t + 3, 2t3 + t2 + 3t + 2, t3 + t2 + 2t + 2 form a basis for P3 .

Subspaces of lRn

Vector Spaces

Vector Spaces and Linear Systems

Vector Spaces Basis and Dimension

Subspaces of lRn Row(Column) Space of a Matrix Orthonormal Bases in lRn

Subspaces of lRn

Vector Spaces

Definition

Let



a11  a21   : am1 be an m × n matrix.

a12 a22 : am2

··· ··· ··· ···

··· ··· ··· ···

··· ··· ··· ···

 a1n a2n   :  amn

Subspaces of lRn

Vector Spaces

Definition

Let



a11  a21   : am1

a12 a22 : am2

··· ··· ··· ···

··· ··· ··· ···

··· ··· ··· ···

 a1n a2n   :  amn

be an m × n matrix.

The rows of A, v1 v2 : vm

= (a11 , a12 , · · · , a1n ) = (a21 , a22 , · · · , a2n ) = (am1 , am2 , · · · , amn ),

considered as vectors in lRn , span a subspace of lRn , called the row space of A.

Subspaces of lRn

Vector Spaces

Definition

Similarly, the columns of A, 

  a11 a12  a21   a22   w1 =   :  , w2 =  : am1 am2





 a1n     , · · · , wn =  a2n  ,  :   amn

considered as vectors in lRm , span a subspace of lRm , called the column space of A.

Vector Spaces

Subspaces of lRn

If A and B are two m Ă— n row equivalent matrices, then the row spaces of A and B are equal.

Subspaces of lRn

Vector Spaces

If A and B are two m × n row equivalent matrices, then the row spaces of A and B are equal.

Example 

1  3 A=  2 −1

 −2 0 3 −4 2 8 1 4   3 7 2 3  2 0 4 −3

Subspaces of lRn

Vector Spaces

If A and B are two m × n row equivalent matrices, then the row spaces of A and B are equal.

Example 

1  3 A=  2 −1

  1 0 2 0 −2 0 3 −4  0 1 1 0 2 8 1 4  ;B=  0 0 0 1 3 7 2 3  0 0 0 0 2 0 4 −3

 1 1   −1  0

Subspaces of lRn

Vector Spaces

Finding a basis

The procedure for finding a basis for the subspace V of lRn given by V = span S, where S = {v1 , v2 , · · · , vk } is a set of vectors in lRn that are given in row form, is as follows.

Step 1. Form the matrix 

 v1  v2   A=  :  vk whose rows are given vectors in S. Step 2. Transform A to reduced row echelon form, obtaining the matrix B. Step 3. The nonzero rows of B form a basis for V.

Subspaces of lRn

Vector Spaces

Example Let S = {(1, −2, 0, 3, −4), (3, 2, 8, 1, 4), (2, 3, 7, 2, 3), (−1, 2, 0, 4, −3)} and let V be the subspace of lR5 given by V = span S. Find a basis for V .

Subspaces of lRn

Vector Spaces

Example Let S = {(1, −2, 0, 3, −4), (3, 2, 8, 1, 4), (2, 3, 7, 2, 3), (−1, 2, 0, 4, −3)} and let V be the subspace of lR5 given by V = span S. Find a basis for V .   1 −2 0 3 −4  3 2 8 1 4   A=  2 3 7 2 3  −1 2 0 4 −3

Subspaces of lRn

Vector Spaces

Example Let S = {(1, −2, 0, 3, −4), (3, 2, 8, 1, 4), (2, 3, 7, 2, 3), (−1, 2, 0, 4, −3)} and let V be the subspace of lR5 given by V = span S. Find a basis for V .     1 −2 0 3 −4 1 0 2 0 1  3  2 8 1 4  1  ;B= 0 1 1 0  A=  2   3 7 2 3 0 0 0 1 −1  −1 2 0 4 −3 0 0 0 0 0

Subspaces of lRn

Vector Spaces

Example Let S = {(1, −2, 0, 3, −4), (3, 2, 8, 1, 4), (2, 3, 7, 2, 3), (−1, 2, 0, 4, −3)} and let V be the subspace of lR5 given by V = span S. Find a basis for V .     1 −2 0 3 −4 1 0 2 0 1  3  2 8 1 4  1  ;B= 0 1 1 0  A=  2   3 7 2 3 0 0 0 1 −1  −1 2 0 4 −3 0 0 0 0 0 Hence, w1 = (1, 0, 2, 0, 1), w2 = (0, 1, 1, 0, 1) and w3 = (0, 0, 0, 1, −1) form a basis for V .

Subspaces of lRn

Vector Spaces

Example Let S = {(1, −2, 0, 3, −4), (3, 2, 8, 1, 4), (2, 3, 7, 2, 3), (−1, 2, 0, 4, −3)} and let V be the subspace of lR5 given by V = span S. Find a basis for V .     1 −2 0 3 −4 1 0 2 0 1  3  2 8 1 4  1  ;B= 0 1 1 0  A=  2   3 7 2 3 0 0 0 1 −1  −1 2 0 4 −3 0 0 0 0 0 Hence, w1 = (1, 0, 2, 0, 1), w2 = (0, 1, 1, 0, 1) and w3 = (0, 0, 0, 1, −1) form a basis for V .

Given that the vector v = (5, 4, 14, 6, 3) is in V , we can write v as a linear combination of the basis obtained above, as follows: v = 5w1 + 4w2 + 6w3

Vector Spaces

Subspaces of lRn

The dimension of the row space of A is called the row rank of A, and the dimension of the column space of A is called the column rank of A.

Vector Spaces

Subspaces of lRn

The dimension of the row space of A is called the row rank of A, and the dimension of the column space of A is called the column rank of A. The row rank and column rank of the m Ă— n matrix A are equal.

Subspaces of lRn

Vector Spaces

Computing for the rank

The procedure for computing the rank A is as follows.

Step 1. Using elementary row operations, transform A to matrix B in reduced echelon form.

Step 2. Rank A = the number of nonzero rows of B.

Subspaces of lRn

Vector Spaces

Nullity A

If A is an m Ă— n matrix, the nullity of A as the dimension of the null space of A, that is, the dimension of the solution space of A~x = ~0.

Subspaces of lRn

Vector Spaces

Nullity A

If A is an m × n matrix, the nullity of A as the dimension of the null space of A, that is, the dimension of the solution space of A~x = ~0.

Example 

1  3 Given A =   2 −1

 −2 0 3 −4 2 8 1 4  .

3 7 2 3  2 0 4 −3

Subspaces of lRn

Vector Spaces

Nullity A

If A is an m × n matrix, the nullity of A as the dimension of the null space of A, that is, the dimension of the solution space of A~x = ~0.

Example 

1  3 Given A =   2 −1

 −2 0 3 −4 2 8 1 4  .

3 7 2 3  2 0 4 −3

If A is an m × n matrix, then rank A+ nullity A = n.

Subspaces of lRn

Vector Spaces

Rank and Singularity

An n Ă— n matrix A is nonsingular if and only if rank A = n.

Subspaces of lRn

Vector Spaces

Rank and Singularity

An n Ă— n matrix A is nonsingular if and only if rank A = n.

Let S = {v1 , v2 , . . . , vn } be a set of n vectors in lRn and let A be the matrix whose rows(columns) are the vectors in S.

Subspaces of lRn

Vector Spaces

Rank and Singularity

An n Ă— n matrix A is nonsingular if and only if rank A = n.

Let S = {v1 , v2 , . . . , vn } be a set of n vectors in lRn and let A be the matrix whose rows(columns) are the vectors in S. Then S is linearly independent if and only if det(A) 6= 0.

Subspaces of lRn

Vector Spaces

Rank and Singularity

An n Ă— n matrix A is nonsingular if and only if rank A = n.

Let S = {v1 , v2 , . . . , vn } be a set of n vectors in lRn and let A be the matrix whose rows(columns) are the vectors in S. Then S is linearly independent if and only if det(A) 6= 0.

The homogeneous system A~x = ~0 of n linear equations in n unknowns has a nontrivial solution if and only if rank A < n.

Subspaces of lRn

Vector Spaces

Definition

A set S = {u1 , u2 , 路 路 路 , uk } in lRn is called orthogonal if any two distinct vectors in S are orthogonal, that is, if ui 路 uj = 0 for i 6= j.

Subspaces of lRn

Vector Spaces

Definition

A set S = {u1 , u2 , · · · , uk } in lRn is called orthogonal if any two distinct vectors in S are orthogonal, that is, if ui · uj = 0 for i 6= j. An orthonormal set of vectors is an orthogonal set of unit vectors. That is, S = {u1 , u2 , · · · , uk } is orthonormal if ui · uj = 0 for i 6= j, and ui · ui = 1 for i = 1, 2, · · · , k.

Subspaces of lRn

Vector Spaces

Definition

A set S = {u1 , u2 , · · · , uk } in lRn is called orthogonal if any two distinct vectors in S are orthogonal, that is, if ui · uj = 0 for i 6= j. An orthonormal set of vectors is an orthogonal set of unit vectors. That is, S = {u1 , u2 , · · · , uk } is orthonormal if ui · uj = 0 for i 6= j, and ui · ui = 1 for i = 1, 2, · · · , k.

Let S = {u1 , u2 , · · · , uk } be an orthogonal set of nonzero vectors in lRn . Then S is linearly independent.

Subspaces of lRn

Vector Spaces

Definition

A set S = {u1 , u2 , · · · , uk } in lRn is called orthogonal if any two distinct vectors in S are orthogonal, that is, if ui · uj = 0 for i 6= j. An orthonormal set of vectors is an orthogonal set of unit vectors. That is, S = {u1 , u2 , · · · , uk } is orthonormal if ui · uj = 0 for i 6= j, and ui · ui = 1 for i = 1, 2, · · · , k.

Let S = {u1 , u2 , · · · , uk } be an orthogonal set of nonzero vectors in lRn . Then S is linearly independent.

An orthonormal set of vectors in lRn is linearly independent.

Subspaces of lRn

Vector Spaces

Dot Product

Definition The dot product or inner product of the n-vectors a = (a1 , a2 , · · · , an ) and b = (b1 , b2 , · · · , bn ) is defined as a · b = a1 b1 + a2 b2 + · · · + an bn =

n X i=1

ai bi

Subspaces of lRn

Vector Spaces

Gram-Schmidt Process Let W be a nonzero subspace of lRn with basis S = {u1 , u2 , · · · , um }.

Then there exists an orthonormal basis T = {w1 , w2 , · · · , wm } for W.

Subspaces of lRn

Vector Spaces

Gram-Schmidt Process Let W be a nonzero subspace of lRn with basis S = {u1 , u2 , · · · , um }.

Then there exists an orthonormal basis T = {w1 , w2 , · · · , wm } for W. Step 1. Let v1 = u1 .

Step 2. Compute the vectors v2 , v3 , · · · , vm , successively, one at a time, by the formula „ « „ « „ « ui · v 1 ui · v2 ui · vi−1 vi = ui − v1 − v2 −· · ·− vi−1 v1 · v1 v2 · v2 vi−1 · vi−1 The set of vectors T ∗ = {v1 , v2 , · · · , vm } is an orthogonal set.

Step 3. Let wi =

vi , 1 ≤ i ≤ m, kvi k

√ where kvi k = vi · vi Then T = {w1 , w2 , · · · , wm } is an orthonormal basis for W.

Subspaces of lRn

Vector Spaces

Exercises 1. Consider the basis S = {(1, 1, 1), (−1, 0, −1), (−1, 2, 3)} for lR3 .

Use the Gram-Schmidt process to transform S to an orthonormal basis for lR3 .

2. Use the Gram-Schmidt process to find an orthonormal basis for the subspace of lR4 with basis {(1, −1, 0, 1), (2, 0, 0, −1), (0, 0, 1, 0)}

Subspaces of lRn

Vector Spaces

Questions?

See you next meeting!