Statistics notes

From raju

Using svd to get principal components

Here are some useful Matlab commands to experiment.

>> x = rand(100,4); c=x'*x, [u, s, v] = svd(x); sum(diag(c)), pca=u*s; c2 = (pca)'* (pca), sum(diag(c2))

c =
   32.3085   25.3927   24.2248   22.5342
   25.3927   36.5004   25.6572   24.9433
   24.2248   25.6572   31.2583   23.5037
   22.5342   24.9433   23.5037   29.5961
ans =
  129.6633
c2 =
  105.7363    0.0000   -0.0000    0.0000
    0.0000    9.0590   -0.0000    0.0000
   -0.0000   -0.0000    8.1320   -0.0000
    0.0000    0.0000   -0.0000    6.7361
ans =
  129.6633

>> subplot(2,2,1); plot(pca(:,1), 'x-'); subplot(2,2,2); plot(pca(:,2), 'x-'); subplot(2,2,3); plot(pca(:,3), 'x-'); subplot(2,2,4); plot(pca(:,4), 'x-')

Notice how sum of the variances of x-variables is same as sum of variances of the pca variables. While the original x-variables have covariances, the pca are mutually orthogonal (i.e. pca covariance matrix is diagonal).

Correlation coefficient for multiple variables

The multiple correlation coefficient generalizes the standard coefficient of correlation for multiple variables. For details see [1]. This article can also be found in the "Multiple Correlation Coefficient" section in [2].

Ref:

  1. https://www.utd.edu/~herve/Abdi-MCC2007-pretty.pdf
  2. Encyclopedia of Measurement and Statistics edited by Neil J. Salkind, 2007.

correlation coefficient

  • Pearson correlation coefficient
measures linear correlation between two variables
https://en.wikipedia.org/wiki/Pearson_correlation_coefficient
  • Spearman's rank correlation coefficient
Pearson correlation coefficient between the rank variables
measures monotonic correlation between two variables
https://en.wikipedia.org/wiki/Spearman%27s_rank_correlation_coefficient
  • Kendall rank correlation coefficient
https://en.wikipedia.org/wiki/Kendall_rank_correlation_coefficient

Type I and Type II errors

Action <math>H_0</math> is actually ...
True False
Do not reject <math>H_0</math> Correct outcome Type II error
Reject <math>H_0</math> Type I error Correct outcome