Statistics notes
From raju
Using svd to get principal components
Here are some useful Matlab commands to experiment.
>> x = rand(100,4); c=x'*x, [u, s, v] = svd(x); sum(diag(c)), pca=u*s; c2 = (pca)'* (pca), sum(diag(c2)) c = 32.3085 25.3927 24.2248 22.5342 25.3927 36.5004 25.6572 24.9433 24.2248 25.6572 31.2583 23.5037 22.5342 24.9433 23.5037 29.5961 ans = 129.6633 c2 = 105.7363 0.0000 -0.0000 0.0000 0.0000 9.0590 -0.0000 0.0000 -0.0000 -0.0000 8.1320 -0.0000 0.0000 0.0000 -0.0000 6.7361 ans = 129.6633 >> subplot(2,2,1); plot(pca(:,1), 'x-'); subplot(2,2,2); plot(pca(:,2), 'x-'); subplot(2,2,3); plot(pca(:,3), 'x-'); subplot(2,2,4); plot(pca(:,4), 'x-')
Notice how sum of the variances of x-variables is same as sum of variances of the pca variables. While the original x-variables have covariances, the pca are mutually orthogonal (i.e. pca covariance matrix is diagonal).
Correlation coefficient for multiple variables
The multiple correlation coefficient generalizes the standard coefficient of correlation for multiple variables. For details see [1]. This article can also be found in the "Multiple Correlation Coefficient" section in [2].
Ref:
- https://www.utd.edu/~herve/Abdi-MCC2007-pretty.pdf
- Encyclopedia of Measurement and Statistics edited by Neil J. Salkind, 2007.
correlation coefficient
- Pearson correlation coefficient
- measures linear correlation between two variables
- https://en.wikipedia.org/wiki/Pearson_correlation_coefficient
- Spearman's rank correlation coefficient
- Pearson correlation coefficient between the rank variables
- measures monotonic correlation between two variables
- https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient
- Kendall rank correlation coefficient
Type I and Type II errors
Action | <math>H_0</math> is actually | ... |
---|---|---|
True | False | |
Do not reject <math>H_0</math> | Correct outcome | Type II error |
Reject <math>H_0</math> | Type I error | Correct outcome |