matrix multiplication

Topics

• linear algebra
• matrices

One of the basic operations in linear algebra is matrix multiplication

$$C_{m\times p}=A_{m\times n}{B}_{n\times p}$$

There are multiple ways to interpret mat-mul apart from the rather conventional dot product of rows of $A$ with columns of $B$.

Rows x Columns

The familiar perspective of taking dot products of rows of $A$ with columns of $B$ to obtain the product $C$. Mathematically,

$$c_{ij}=\sum _{k=1}^n{a}_{ik}{b}_{kj}$$

Matrix x Columns

The interpretation here is that, each column of $B$ specifies a linear combination of columns of $A$, to produce the columns of $C$. So, if we want to rearrange the columns of a matrix, multiply it by another matrix on the right.

Example

Find the transformation matrix that changes the sign of $1^{st}$ column of $A$ and swaps the $2^{nd}$ and $3^{rd}$ columns. (Note $A$ is a $3\times 3$ matrix, say)
Answer:

$$B=\left[\begin{array}{ccc}-1& 0& 0\\ 0& 0& 1\\ 0& 1& 0\end{array}\right]$$

Rows x Matrix

$AB$ can also be viewed as multiplying each row of $A$ by the matrix $B$ on the right. Multiplying a row vector by a matrix on the right produces another row vector. Each row of $A$ specifies a linear combination of rows of $B$ to produce rows of $C$. So, if we want to rearrange the rows of a matrix, multiply it by another matrix on the left.

Example

Find the transformation matrix that adds the third row of $B$ to two times the first row and leaves other rows untouched (P.S: similar to steps done during Gaussian Elimination).
Answer:

$$A=\left[\begin{array}{ccc}2& 0& 1\\ 0& 1& 0\\ 0& 0& 1\end{array}\right]$$

Columns x Rows

The key to this perspective is to observe:

• Elements in column $i$ of $A$ only multiply elements in row $j$ of $B$.
• A column times a row vector, sometimes denoted $x{y}^T$ , is an outer product and produces a rank-$1$ matrix (as the matrix has one independent vector and others are multiples of it).
  So, from this perspective, we could write:

$$\begin{array}{rl}AB& =\sum _{k=1}^3\text{(column k of A)(row k of B)}\\ & =\sum _{k=1}^{3}A[:,k]B[k,:{]}^T\end{array}$$

Example

If

$$A=\left[\begin{array}{cc}a& b\\ c& d\end{array}\right]$$

and

$$B=\left[\begin{array}{cc}e& f\\ g& h\end{array}\right]$$

We then have:

$$\begin{array}{rl}AB& =\left[\begin{array}{cc}ae+bg& af+bh\\ ce+dg& cf+dh\end{array}\right]\\ & =\left[\begin{array}{cc}ae& af\\ ce& cf\end{array}\right]+\left[\begin{array}{cc}bg& bh\\ dg& dh\end{array}\right]\\ & =\left[\begin{array}{c}a\\ c\end{array}\right]\left[\begin{array}{cc}e& f\end{array}\right]+\left[\begin{array}{c}b\\ d\end{array}\right]\left[\begin{array}{cc}g& h\end{array}\right]\end{array}$$

Thus, $AB$ can be written as the sum of the corresponding column-row outer products.

Related