Main content

Course: Staging content lifeboat > Unit 21

Lesson 3: Prerequisites

Linear algebra vocabulary refresher

Linear algebra in multivariable calculus

We just finished talking about all the different ways we can visualize multivariable functions. As a final stop before we start to focus more on the calculus part of multivariable calculus, let's discuss a uniquely important tool that we'll use throughout the course. At its core, linear algebra gives us a way to describe higher dimensions compactly. This is indispensable in calculus beyond a single variable, or in other words, beyond the first dimension. A foundation in linear algebra helps us really see the beauty in multivariable calculus.

This article won't go into too much detail, because there's already a whole course for linear algebra on Khan Academy. Instead, it will present a review of key terms and concepts that will appear again and again in multivariable calculus. If you don't recognize a concept, read on to see a brief discussion of it and its multivariable applications. If you still don't understand the idea, there are links to Khan Academy content that will help you learn.

Vectors

Vectors are the building blocks of everything multivariable. We use them when we want to represent a coordinate in higher-dimensional space or, more generally, to write a list of anything. These lists could be functions that can describe moving coordinates or even transformations of the entire input space.

There are lots of notations for vectors, but here are the three we'll use most in this course. Sometimes we write a little arrow on top of variables that are vectors to distinguish them, although people also often omit it.

\begin{aligned} \vec{v} & = (1, 2, 3) \\ \vec{v} & = 1 \hat{ı} + 2 \hat{ȷ} + 3 \hat{k} \\ \vec{v} & = [\begin{array}{c} 1 \\ 2 \\ 3 \end{array}] \end{aligned}

All the examples above are of 3D vectors, but a vector can be

n

dimensional in general, where

n

is any positive integer.

The first notation is the familiar triple of points, which is common because it generalizes to all dimensions. The second notation, on the other hand, only works in 2D and 3D. The symbol

\hat{ı}

(pronounced i-hat) is the unit

x

vector, so

\hat{ı} = (1, 0, 0)

. Similarly,

\hat{ȷ} = (0, 1, 0)

and

\hat{k} = (0, 0, 1)

. The third notation is matrix notation and can be generalized to as many dimensions as we want by adding more rows.

The two basic operations of vectors are addition and scalar multiplication.

\begin{aligned} \vec{a} & = (0, - 1, - 2) \\ \vec{b} & = (1, 2, 3) \\ \vec{a} + \vec{b} & = (0 + 1, - 1 + 2, - 2 + 3) = (1, 1, 1) \\ 2 \vec{a} & = (2 \cdot 0, 2 \cdot (- 1), 2 \cdot (- 2)) = (0, - 2, - 4) \\ 2 \vec{a} + \vec{b} & = (0 + 1, - 2 + 2, - 4 + 3) = (1, 0, - 1) \end{aligned}

In multivariable calculus, we might write

(f_{0} (t), f_{1} (t), f_{2} (t))

to denote a vector whose

x

-component is the function

f_{0}

evaluated at

t

, whose

y

-component is

f_{1} (t)

, and whose

z

-component is

f_{2} (t)

. There are new ways to think about vectors like this that we'll see as we progress further into multivariable calculus.

Learn more about vectors here.

Beyond addition and scalar multiplication, there are two more important operations between vectors.

Dot products

The dot product is written

\vec{v} \cdot \vec{w}

and gives a number. This number is a measure of how long the vectors are and also how much the two vectors point in the same direction. If the vectors are directly perpendicular, then the dot product is zero. The dot product is largest when the two vectors are parallel.

The definition of the dot product is given below, where

θ

is the angle between the two vectors. In words, it says that the dot product between two vectors is the product of their lengths times the cosine of the angle between them.

\vec{v} \cdot \vec{w} = ‖ v ‖ ‖ w ‖ \cos (θ)

The more common way to find the dot product requires knowing the components of the vectors. For two vectors

\vec{v} = (v_{0}, \dots, v_{n})

and

\vec{w} = (w_{0}, \dots, w_{n})

, we have:

\vec{v} \cdot \vec{w} = v_{0} w_{0} + \dots + v_{n} w_{n}

Here are a few examples.

\begin{aligned} \vec{v} = 4 \hat{ı} - 1 \hat{ȷ} + 2 \hat{k} \\ \vec{w} = 1 \hat{ı} + 2 \hat{ȷ} + 4 \hat{k} \\ \vec{v} \cdot \vec{w} = 4 \cdot 1 + (- 1) \cdot 2 + 2 \cdot 4 = 10 \\ (4, 5) \cdot (- 5, 4) = 4 \cdot (- 5) + 5 \cdot 4 = 0 \\ (0, 5, - 4) \cdot (9, 2, 3) = 0 \cdot 9 + 5 \cdot 2 + (- 4) \cdot 3 = - 2 \end{aligned}

Notice how

(4, 5) \cdot (- 5, 4) = 0

. If we graph these two vectors, we see that they are indeed perpendicular. This is the work of the

\cos (θ)

in the definition, because

\cos (\frac{π}{2}) = 0

Learn more about dot products here.

Cross products

The dot product is defined in any number of dimensions, but the cross product only works if both vectors are three dimensional. The cross product is written

v \times w

and gives a vector.

This vector has two special properties: it is perpendicular to both

v

and

w

, and its magnitude corresponds to the area of the parallelogram formed by

v

and

w

If two vectors are parallel, then their cross product is zero. The cross product is largest when the two vectors are perpendicular. In this sense, the cross product complements the dot product. The dot product is zero for perpendicular vectors, while the cross product is zero for parallel vectors.

Learn more about the connection between the dot and cross product here.

The magnitude of the cross product is

| v \times w | = ‖ v ‖ ‖ w ‖ \sin (θ)

, where

θ

is again the angle between the two vectors. In words, it says that the magnitude is the product of the lengths of the two vectors times the sine of the angle between them.

The formula for the cross product of two vectors is quite a mouthful and should not be memorized. Think of the cross product always as perpendicular to its two input vectors and with area equal to that of the parallelogram they form.

\begin{aligned} \vec{v} & = (v_{1}, v_{2}, v_{3}) \\ \vec{w} & = (w_{1}, w_{2}, w_{3}) \\ \vec{v} \times \vec{w} & = (v_{2} w_{3} - v_{3} w_{2}, v_{3} w_{1} - v_{1} w_{3}, v_{1} w_{2} - v_{2} w_{1}) \end{aligned}

Notice, for example, that

\hat{ı} \times \hat{ȷ} = \hat{k}

Learn more about cross products here.

Matrices

When we want to talk about lots of vectors all at once, we can use a matrix. Think of each column as its own vector, except now we can represent three vectors with one mathematical object. (Sometimes we think of the rows as vectors instead of the columns.)

[\begin{matrix} 1 & 4 & 7 \\ 2 & 5 & 8 \\ 3 & 6 & 9 \end{matrix}]

Matrices have a concept of addition and multiplication, although it's far more useful to think of these operations as transformations than as the formulas we use to compute them.

Learn more about matrices as transformations here.

Nonetheless, we will need to know how to compute with matrices in order to solve problems. Always remember that matrices still represent transformations encoded with numbers.

Learn more about computing with matrices here.

Inverse matrices

Given that we can multiply matrices, there is one matrix that stands out. We call it

I

, the identity matrix.

I = [\begin{matrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{matrix}]

The identity matrix

I

isn't only defined for

3 \times 3

matrices. In general,

I

is the

n \times n

matrix with ones on its diagonal and zeros everywhere else.

The matrix

I

is special because it is the only matrix that makes

I A = A I = A

for any

n \times n

matrix

A

. The identity matrix

I

is similar to the number

1

when we're in matrix-land.

We know that we can take any nonzero real number

x

and multiply it by

x^{- 1}

to get

x x^{- 1} = x^{- 1} x = 1

. A natural question might be, "Can we do the same for matrices?"

The answer is that usually we can. If we have a matrix

A

and it has an inverse

A^{- 1}

, then we call

A

invertible. The inverse of

A

must satisfy

A A^{- 1} = A^{- 1} A = I

Taking a step back from all the symbols, what is an inverse matrix? What does it mean to multiply a matrix by its inverse?

If you haven't seen matrices as transformations before, look at this first. For example, the matrix $I$ ‍ is the identity matrix whose transformation is to leave everything the same.

Here's an example of inverse matrices:

The matrix

2 I

stretches everything out by a factor of

2

, and the matrix

0.5 I

squeezes everything in by a factor of

2

. When we apply

2 I

and

0.5 I

successively, we stretch then squeeze by equal amounts, so the net transformation is just the identity transformation

I

. Therefore,

(2 I)^{- 1} = 0.5 I

Now, an arbitrary matrix

T

has a corresponding transformation. We start with

I

, the "do nothing" matrix. Then

T (I)

means start from "do nothing," then transform by

T

. If

T

is invertible, then

T^{- 1} (T I)

means start from "do nothing," then transform by

T

, then perform the exact transformation that undoes

T

. This is again

I

, the "do nothing" transformation.

In light of the transformation mindset, the inverse of a matrix

A

is the matrix whose transformation exactly undoes the effect of

A

Determinants

Let's return to the question of when a matrix has an inverse. Not all matrices are invertible. For example, try as we might, we will never find an inverse for the matrix

A

A = [\begin{matrix} - 4 & 0 & - 2 \\ 2 & 0 & 1 \\ - 2 & 3 & 5 \end{matrix}]

Why not? Every nonzero number has an inverse, but there is no number that will multiply with

0

to give

1

. Many matrices act like

0

does on the number line. We can even think of numbers as

1 \times 1

matrices.

Specifically, when a matrix acts as a transformation, we can talk about it stretching things out or squeezing them in. Imagine that we applied a linear transformation to a cube, for example. It'll get warped somehow into a rectangular prism, and its volume will perhaps be larger than at the start.

This expansion and contraction is so important that we have a special function that tells us how much a certain matrix stretches or squeezes space. We call this function the determinant.

When we consider what properties the inverse of a matrix must have, a big one is that it should exactly reverse the stretching or squeezing the original matrix caused. This is possible for almost all matrices, but a select few shrink space infinitely.

Imagine the cube from before not just getting warped, but getting completely flattened into the

x y

-plane. We have lost information that was essential to that cube, so we cannot build rebuild it exactly.

Where else have you encountered a kind of fundamental loss of information? Think about derivatives.

\frac{d}{d x} [5] = 0 = \frac{d}{d x} [2]

There is no way to piece together exactly what a function was given its derivative, because the derivative inherently destroys information when it sets constants to zero. That's why integrals need a constant

C

at the end.

While we're on the topic of derivatives, I want to reiterate a few properties of derivatives.

\begin{aligned} (f (x) + g (x))^{'} & = f^{'} (x) + g^{'} (x) \\ (a f (x))^{'} & = a f^{'} (x) \end{aligned}

We're getting more than our feet wet now, but these are exactly the properties that are required for a linear transformation. In fact, the derivative is a linear transformation that brings differentiable functions to the space of differentiable functions. It even has a matrix.

D = [\begin{matrix} 0 & 1 & 0 & 0 & \dots \\ 0 & 0 & 2 & 0 & \dots \\ 0 & 0 & 0 & 3 & \dots \\ ⋮ & ⋮ & ⋮ & ⋮ & ⋱ \end{matrix}]

Here we assume that our functions are all polynomials. For example, we would write

f (x) = 3 x^{2} - 2 x + 1

as a column vector with an infinite tail of zeros.

f = [\begin{matrix} 1 \\ - 2 \\ 3 \\ 0 \\ 0 \\ ⋮ \end{matrix}]

Then the derivative magic:

D [\begin{matrix} 1 \\ - 2 \\ 3 \\ 0 \\ ⋮ \end{matrix}] = [\begin{matrix} - 2 \\ 6 \\ 0 \\ 0 \\ ⋮ \end{matrix}]

As expected, the derivative is

f^{'} (x) = 6 x - 2

As a challenge, if you accept that you'll never be able to know the constant term of an integral, can you find a similar matrix for it?

Another way to say this is that matrices with determinant zero are not invertible.

We will need to compute determinants in multivariable calculus, so it's worthwhile to know the formula. There is a good explanation from linear algebra for why the determinant is calculated as it is. Until then, try to always think of determinants in terms of how they stretch and squeeze space.

Learn about calculating determinants here.

Onward!

Now you know the linear algebra that is essential to fruitfully understanding multivariable calculus. Whenever a problem involving one of these topics is difficult, always come back to what the object represents.

Vectors represent directions in space. Dot products are a measure of whether two vectors are pointing in the same direction. Cross products give perpendicular vectors. Matrices represent linear transformations of space. Inverse matrices are the opposite transformation of a certain matrix. Determinants give the scaling factor of a matrix.

Want to join the conversation?

Sort by:

No posts yet.