Lesson 1: Matrix Review

Lesson 1: Matrix Review
Introduction
Why Matrix Algebra?
Univariate Statistics: Are concerned with random scalar variables, Y.
Example: Y may denote the daily vitamin C intake of a randomly selected
woman in the USDA women's nutrition survey. Multivariate Statistics: Are concerned with random vectors, Y.
Example: For a randomly selected woman in the USDA women's nutrition
survey, we may have:
[pic]
Note: Each of the elements of Y is a random variable.
[pic]
Operations carried out on scalars in univariate statistics are also carried
out by analogous operations on vectors and matrices in multivariate
statistics.
Example: Consider the sample mean of a random variable Y:
[pic]
For multivariate statistics, we may compute the mean of a random vector Y
where:
[pic]
Here, the elements of the sample mean vector are equal to the sample means
of the individual variables. The sample mean vector may be computed by
separate calculations of the sample means of the component variables. Or,
equivalently, we may add all of the data vectors together then divide each
element of the resulting vector by the sample size n. Learning objectives & outcomes The objective of this lesson is to review basic matrix operations necessary
to understand multivariate statistics. These operations include:
. Transpose
. Addition
. Multiplication
. Identity Matrix
. Matrix Inverse
[pic] Matrix Operations In the following, we will consider n x m matrices of the form:
[pic]
A vector is an n x 1 matrix, for instance:
[pic]
[pic] Matrix Transpose: [pic]
Example: Symmetric Matrices Note that a matrix A is symmetric if A' = A; that is, if aij = aij.
Important examples of symmetric matrices in multivariate statistics include
the variance-covariance matrix and the correlation matrix. These shall be
defined when we consider descriptive statistics.
Examples:
|This matrix below |This matrix below |
|[pic] |[pic] |
|is symmetric. |is not symmetric. |
Addition The sum of two matrices:
[pic]
[pic]
Here the notation "n×m" means that each of the matrices A, B, and C has n
rows and m columns. Two matrices may be added if and only if they have
identical numbers of rows and they have identical numbers of columns.
Matrices are added by summing the corresponding columns of each matrix.
Thus the ijth column of C is obtained by summing the ijth elements of A and
B.
Example:
[pic]
[pic] Multiplication The product of two matrices:
[pic]
[pic]
Here the number of columns in A must equal the number of rows in B. Note:
In general, AB ? BA.
Example:
[pic]
[pic] The Identity Matrix The identity matrix has ones in the diagonal and zeros in the off-diagonal
elements:
[pic]
It is called the identity matrix since multiplication of any matrix A by
the identity matrix yields the original matrix A:
AI = IA = A Matrix Inverse Square matrices only: A-1 is the inverse of A if
AA-1 = I
For 2 x 2 Matrices, we have the formula:
[pic]
Example:
[pic]
Always check your work!
[pic] General n x n Matrices: To obtain an algorithm for inverting general n x n matrices, we must review
three elementary row operations:
1. Exchange two rows.
2. Multiply the elements of a row by a constant.
3. Add a multiple of another row to the given row. Obtaining the Inverse of Matrix A To obtain the inverse of a n x n matrix A :
Step 1: Create the partitioned matrix ( A I ) , where I is the identity
matrix.
Step 2: Perform elementary row operations on the partitioned matrix with
the objective of converting the first part of the matrix to the identity
matrix.
Step 3: Then the resulting partitioned matrix will take the form ( I A-1 )
Step 4: Check your work by demonstrating that AA-1 = I.
Below is a demonstration of this process: Summary In this lesson we learned how to carry out basic matrix operations:
. Transpose
. Addition
. Multiplication
. Inverse
In addition, you should know the definitions of:
. Symmetric Matrix
. Identity Matrix Lesson 2: Graphical Display of Multivariate Data
Introduction One of the first steps in the analysis of any dataset is an Exploratory
Data Analysis (EDA), including the graphical display of the data.
Why do we look at graphical displays of the data? Your reasons might
include to:
. suggest a plausible model for the data,
. assess validity of model assumptions,
. detect outliers, or
. suggest plausible normalizing transformations
Many multivariate methods assume that the data are multivariate normally
distributed. Exploratory data analysis through the graphical display of
data can be used to assess the normality of the data. If evidence is found
that the data are not normally distributed, then graphical methods can be
applied to determine appropriate normalizing transformations for the data. Learning objectives & outcomes The objectives of this lesson are:
. Introduce graphical methods for summarizing multivariate data
including histograms, matrices of scatterplots, and rotating 3-
dimensional scatterplots;
. Produce graphics using SAS interactive data analysis;
. Understand when transformations of the data should be applied, and
what specific transformations should be considered;
. Learn how to identify unusual observations (outliers), and understand
issues regarding how outliers should be handled if they are detected. Graphical Methods
Example: USDA Women's Health Survey Let's take a look at an example. In 1985, the USDA commissioned a study of
women's nutrition. Nutrient intake was measured for a random sample of 737
women aged 25-50 years. The following variables were measured:
. Calcium(mg)
. Iron(mg)
. Protein(g)
. Vitamin A(?g)
. Vitamin C(mg)
Here are the different ways we could take a look at this data graphically
using SAS's Interactive Data Analysis tools. Univariate Cases: Using Histograms we can:
. Assess Normality
. Find Normalizing Transformations
. Detect Outliers
Here we have a histogram for daily intake of calcium. Note that the data
appear to be skewed to the left, suggesting that calcium is not normally
distributed. This suggests that a normalizing transformation should be
considered.
[pic]
Common transformations include:
. Square Root (often used with counts data)
. Quarter Root
. Log (either natural or base 10)
The square root transformation is the weakest of the above transformations,
while the log transformation is the strongest. In practice, it is generally
a good idea to try all three transformations to see which appears to yield
the most symmetric distribution.
The following shows histograms for the raw data (calcium), square-root
transformation (S_calciu), quarter-root transformation (S_S_calc), and log
transformation (L_calciu). With increasingly stronger transformations of
the data, the distribution shifts from being skewed to the left to being
skewed to the right. Here, the square-root transformed data is still
slightly skewed to the left, suggesting the that the square-root
transformation is not strong enough. In contrast, the log-transformed data
are skewed to the right, suggesting that the log transformation is too
strong. The quarter-root transformation results in the most symmetry
distribution, suggesting that this transformation is most appropriate.
[pic]
In practice, histograms should be plotted for each of the variables, and
transformations should be applied as needed. Bivariate Cases: Using Scatter Plots we can:
. Describe relationships between pairs of variables
. Assess linearity
. Find Linearizing Transformations
. Detect Outliers
Here we have a scatterplot in which calcium is plotted against iron. This
plot suggests that daily intake of calcium tends to increase with
increasing daily intake of iron. If the data are bivariate normally
distributed, then the scatterplot should be approximately elliptical in
shape. However, the points appear to fan out from the origin, suggesting
that the data are not bivariate normal.
[pic]
After applying quarter-root transformations to both calcium and iron, we
obtain a scatter of point that appears to be more elliptical in shape.
Moreover, it appears that the relationship between the transformed
variables is approximately linear. The point in the lower left-hand corner
appears to be an outlier, or unusual observation. Upon closer examination,
this woman reports zero daily intake of iron. Since this is very unlikely
to be correct, we might justifiably remove this observation from the data
set.
[pic]
Outliers:
Note that it is not appropriate to remove an observation from the data just
because it is an outlier. Consider, for example, the ozone hole in the
Antarctic. For years, NASA had been flying polar-orbiting satellites
designed to measure ozone in the upper atmosphere without detecting an
ozone hole. Then, one day, a scientist visiting the Antarctic pointed an
instrument straight-up into the sky, and found evidence of an ozone hole.
What happened? It turned out that the software used to process t