Understanding behind the scene calculation for eigen value and eigen vector
PCA Stands for principal component analysis. It helps in dimensionality reduction or feature engineering.
PCA is very important concept in machine learning. As machine learning depends upon the data and success of model too depends upon data. In current world, we have abundance of data and we can’t feed all data to model directly. It becomes cumbersome to have umpteen number of independent columns and it could be redundancy of features in the training data. There is must to bring down the features based on experts advice or your learning of data. PCA comes as rescuer which learns from the data and helps us to find the most important independent features in the given data. In simple words, it will provide us the true representative of the whole data. It is like MLA voted by public for particular area represents the whole area. Similarly, features considered important by PCA tells us that how much % it represents the whole data. Let us explore to understand the inside working of PCA.
Starting Matrix A= {2,6} {13,9}
Step1 : Lamba I = Lambda where I is identity matrix {1,0}{0,1}. So Multiplying lambda with I gives {Lambda,0} {0,Lambda}
Step2 : Calculate the subtraction of matrix and Lambda I. We will get in our case: {2-Lambda, 6} {13, 9-Lambda}
Step3 : Calculate determinant of resultant matrix of step2. And, we will get Square of Lambda -11Lambda + 60
Step4 : Assume equation in step3 is equal to zero and calculate value of Lamba. Solving the above quadratic equation, we will get Lambda = 15,-4. These two values are eigenvalue for the given matrix.
Step5 : Need to calculate eigenvector for corresponding eigenvalue. Input Lambda = 15 in step2 equation. Output will be {-13,6} {13,6}
Step6 : Multiple step5 matrix with mean of x and consider it equal to 0. Therefore, {-13,6}{13,6} * {x1,x2} = 0. Calculating this we will get the equation: -13×1+6×2 =0
Step7 : Following the resultant equation in step6, we can get x1 = (6/13) x2. If we assume x2=13, x1 will become 6. Or, take x1 = 0.46 x2 as [6/13=0.46]. In that…
