Algebra defines, roughly, relationships. See: http://blog.ruofeidu.com/gradient-circulation-laplacian-divergence-jacobian-hessian-trace/

## Matrix Form for Linear Regression

For the regression model $$Y = AX + B$$, the coefficients of the least squares regression line are given by the matrix equation

$$A = (X^T X)^{-1} X^T Y$$

and the sum of squared error is

$$B^T B$$

# Definitions

- Shape: Descriptions like “upper-triangular”, “symmetric”, “diagonal” are the shape of the matrix, and influence their transformations.
- rank(A) is the dimension of the vector space generated (or spanned) by its columns, which can be computed by reducing the matrix to row echelon form.
- rank(AB) <= min(rank(A), rank(B))

- tr(A) is the sum of the elements on the main diagonal
- tr(A+B) = tr(A) + tr(B)

- Gradient is a multi-variable generalization of the derivative
- Divergence of a three-dimensional vector field is the extent to which the vector field flow behaves like a source at a given point
- Jacobian matrix is the matrix of all first-order partial derivatives of a vector-valued function.
- Hessian matrix or Hessian is a square matrix of second-order partial derivatives of a scalar-valued function, or scalar field.

# Transformation

A linear transformation $$T:R_n→R_m$$ is a mapping from n-dimensional space to m-dimensional space. Such a linear transformation can be associated with an $m\times n$ matrix.

- Reflection in y
- $$T(x, y) = (-x, y)$$
- $$\begin{pmatrix} -1 & 0 \\ 0 & 1 \end{pmatrix}$$

- Shears
- $$T(x, y) = (x + ky, y)$$
- $$\begin{pmatrix} 1 & k \\ 0 & 1 \end{pmatrix}$$

- Rotations
- $$\begin{pmatrix} \cos \theta & -\sin \theta \\ \sin \theta & \cos \theta \end{pmatrix}$$

# Determinant

The determinant is the “

size” of the output transformation.If the input was a unit vector (representing area or volume of 1), the determinant is the size of the transformed

areaorvolume.A determinant of 0 means matrix is “destructive” and cannot be reversed (similar to multiplying by zero: information was lost).

## Inverse

If A is an invertible matrix, then

$$A^{-1} = \frac{1}{det(A)} adj(A)$$

# Eigenvector and Eigenvalue

The eigenvector and eigenvalue represent the “axes” of the transformation.

Consider spinning a globe: every location faces a new direction, except the poles.

An “

eigenvector” is an input that doesn’t change direction when it’s run through the matrix (it points “along the axis”).And although the direction doesn’t change, the size might.

The

eigenvalueis the amount the eigenvector is scaled up or down when going through the matrix.

### Cramer’s Rule

If a system of linear equations in variables has a coefficient matrix with

a nonzero determinant then the solution of the system is

$$x_1 = \frac{det(A_1)}{det(A)}, x_2 = \frac{det(A_2)}{det(A)}, x_n = \frac{det(A_n)}{det(A)}$$

### Area of a Triangle in the XY-Plane

$$Area = \frac{1}{2} \begin{vmatrix} x_1 & y_1 & 1 \\ x_2 & y_2 & 1 \\ x_3 & y_3 & 1 \end{vmatrix} $$

### Finding the Volume of a Tetrahedron

$$\frac{1}{6} \begin{vmatrix} x_1 & y_1 & z_1 & 1 \\ x_2 & y_2 & z_2 & 1 \\ x_3 & y_3 & z_3 & 1 \\ x_4 & y_4 & z_4 & 1 \end{vmatrix} $$

## Cross Product

$$ u \times v = \begin{pmatrix} i & j & k \\ u_1 & u_2 & u_3 \\ v_1 & v_2 & v_3 \end{pmatrix} $$

for example, in $R^3$

$$u \times v = (u_2 v_3 – u_3 v_2) i – (u_1v_3 – u_3v_1)j + (u_1v_2 – u_2v_1)k$$

In physics, the cross product can be used to measure torque—the moment of a force about a point as shown in the figure below:

When the point of application of the force is the moment of about is given by

$$M = AB \times F $$

where represents the vector whose initial point is and whose terminal point is The magnitude of the moment measures the tendency of to rotate counterclockwise about an axis directed along the vector M.

### Four points and are coplanar if and only if

$$det \begin{vmatrix} x_1 & y_1 & z_1 & 1 \\ x_2 & y_2 & z_2 & 1 \\ x_3 & y_3 & z_3 & 1 \\ x_4 & y_4 & z_4 & 1 \end{vmatrix} = 0 $$

The volume of a parallelepiped spanned by the vectors aa, bb and cc is the absolute value of the scalar triple product (a×b)⋅c(a×b)⋅c.

### Three-Point Form of the Equation of a Plane

An equation of the plane passing through the distinct points and is given by

$$det \begin{vmatrix} x & y & z & 1 \\ x_1 & y_1 & z_1 & 1 \\ x_2 & y_2 & z_2 & 1 \\ x_3 & y_3 & z_3 & 1 \\ \end{vmatrix} $$

# References

- http://blog.ruofeidu.com/gradient-circulation-laplacian-divergence-jacobian-hessian-trace/
- http://homepage.ntu.edu.tw/~jryanwang/course/Mathematics%20for%20Management%20(undergraduate%20level)/Applications%20in%20Ch3.pdf
- https://betterexplained.com/articles/linear-algebra-guide/

## Leave a Reply