We have covered projection in Dot Product. Now, we will take deep dive into projections and projection matrix.

Consider two vectors in 3D vector space: a and b

Say, we have a point R on vector a that is closest to vector b

We can construct a vector to the point R having the same direction as vector a.

As the new vector r shares the direction with vector a, it could be represented as a product of vector a with some scalar quantity (in this case, X).

Assuming the distance between point R and vector b is (b-r).

From the figure, we can see that e is perpendicular to a.

So

See orthogonal vectors

This equation could be simplified further for the value of X

We know that

Scalar multiplication is commutative

Now we will simplify this equation

Coming back to the figure

If vector r is a projection of vector b on vector a.

Then vector r could be represented as

P is Projection matrix

and

## Properties of Projection Matrix

The projection matrix has some amazing properties like

Projection matrix is symmetric

and

Square of projection matrix is itself

The matrices that having this property are called Idempotent Matrices.

So, if we project a vector twice the result will be same as with projecting once.

## Why use projections?

Suppose we have a matrix A comprising of two column vectors a1 and a2.

Column Space of A

Now, we have to find solution of

such that vector b does not lie in column space of matrix A.

Vector b lies outside the column space of A

Looks like there are no solutions to this equation because there is no way vector b could be represented as a linear combination of a1 and a2.

But, if we project vector b onto the column space of A

Vector p is projection of vector b on the column space of matrix A

Vectors p, a1 and a2 all lie in the same vector space.

Therefore, vector p could be represented as a linear combination of vectors a1 and a2.

x1 and x2 are the coefficients to be multiplied with vectors

This could be broken down in the form of matrices

We can solve for matrix x and we may get a solution to the problem we previously thought was unsolvable.

Read Part 18 : Norms

You can view the complete series here

Connect with me on LinkedIn.