A system with 𝑚 equations and 𝑛 variables

\[ \begin{aligned} a_{1,1}\,x_1 + \cdots +a_{1,n}\,x_n & = y_1\\ a_{2,1}\,x_1 + \cdots +a_{2,n}\,x_n & = y_2\\ ⋮\qquad ⋱ \qquad ⋮\quad &= ⋮\\ a_{m,1}\,x_1 + \cdots + a_{m,n}\,x_n & = y_m \end{aligned} \]

we know all \(y_i\) and \(a_{i,j}.\) We want to find \(x_j\)

As matrices

\[ \begin{bmatrix} a_{1,1} &\cdots & a_{1,n}\\ ⋮ & ⋱ & ⋮ \\ a_{m,1} & \cdots & a_{m,n} \end{bmatrix} \begin{bmatrix} x_1 \\ ⋮ \\ x_n \end{bmatrix} = \begin{bmatrix} y_1 \\⋮\\ y_m \end{bmatrix} \]

\[\underbrace{𝐀}_{m×n}\,\underbrace{𝐱}_{n×1}=\underbrace{𝐲}_{m×1}\]

When can we solve the system of equations?

Sometimes there is no solution

Sometimes there are infinite solutions

We want to have a single solution

Can we solve it when \(m=1\)?

One equation and \(n\) variables \[a_{1}\,x_1 + a_{2}\,x_2 = y\] Or, equivalently, \[𝐚^T𝐱 = y\] for \(𝐚=(a_1, a_2)^T\) and \(𝐱=(x_1, x_2)^T\)

A solution

Let’s choose \(x_2=1,\) so \(y - a_{2} = a_{1}\,x_1\) and \[ \frac{y- a_{2}}{a_{1}} = x_1\] and therefore \[𝐱_*= \begin{bmatrix} (y - a_{2})/a_{1} \\ 1 \end{bmatrix}\] is a solution of \(y=𝐚^T𝐱\)

More solutions

Any vector \(𝐱'\) orthogonal to \(𝐚\) will satisfy \[𝐚^T𝐱'=0\] and then, for any number \(α\) \[\begin{aligned} 𝐚^T(𝐱_*+α𝐱')&=𝐚^T𝐱_* + α\,𝐚^T𝐱'\\ &= y + α\,0\\ &=y \end{aligned}\] so \((𝐱_*+α𝐱')\) is also a solution, for any \(α\)

There are always vectors orthogonal to \(𝐚\)

If there is a vector \(𝐱'≠0\) orthogonal to \(𝐚\) then we have an infinite number of solutions

For instance, \(𝐱'=(-a_2, a_1)^T\) is orthogonal to \(𝐚\)

This will always be the case when \(m<n\)

Hyperplanes

Notice that the equation \[𝐚^T𝐱=0\] defines

• a line (in 2D),
• a plane (in 3D) or
• an hyperplane (in higher dimensions)

When 𝑚>𝑛

When there are more rows than columns, we have too many conditions

In most cases these conditions are incompatible

For example

\[ \begin{aligned} 1\,x_1 + 2\,x_2 & = 4\\ 3\,x_1 + 4\,x_2 & = 10\\ 5\,x_1 + 6\,x_2 & = 14\\ \end{aligned} \]

Square matrices: 𝑚=𝑛

When \(m<n\) if we have at least one solution, then we will have an infinite number of solutions

When \(m>n\) in general we do not have any solution

When \(m=n\) in general we will have a single solution

… but see next slides

What about this system?

This seems to be a square matrix \[ \begin{aligned} y_1 & = a_{1,1}\,x_1 + a_{1,2}\,x_2\\ y_1 & = a_{1,1}\,x_1 + a_{1,2}\,x_2 \end{aligned} \] The second equation is the same as the first one, so it is redundant

It seems as we have two equations, but there is only one

What about this other system?

\[ \begin{aligned} y_1 & = a_{1,1}\,x_1 + a_{1,2}\,x_2\\ α\,y_1 & = α\,a_{1,1}\,x_1 + α\,a_{1,2}\,x_2 \end{aligned} \] The second equation is the same as the first one, barely disguised

In particular if \(α=0\) we have \[ \begin{aligned} y_1 & = a_{1,1}\,x_1 + a_{1,2}\,x_2\\ 0 & = 0\,x_1 + 0\,x_2 \end{aligned} \] which is just the first equation

Sometimes they are better disguised

We can have these equations \[ \begin{aligned} y_1 & = a_{1,1}\,x_1 + a_{1,2}\,x_2 + a_{1,3}\,x_3\\ y_2 & = a_{2,1}\,x_1 + a_{2,2}\,x_2 + a_{2,3}\,x_3\\ y_1+y_2 & = (a_{1,1} + a_{2,1})\,x_1 + (a_{1,2} + a_{2,2})\,x_2 + (a_{1,2} + a_{2,3})\,x_3 \end{aligned} \]

The last equation is a combination of the first two equations

Combining vectors

It could be \[ \begin{aligned} α\,y_1 + β\,y_2 & = (α\,a_{1,1} + β\,a_{2,1})\,x_1 + (α\,a_{1,2} + β\,a_{2,2})\,x_2 + (α\,a_{1,2} + β\,a_{2,3})\,x_3 \end{aligned} \] we say that the third equation is a linear combination of the first two equations

count “distinct” rows

Two visions of matrix vector multiplication