Overview

In this lesson, we'll learn about the components of vectors, as well as some basic vector operations.

Outcomes

After completing this lesson, you'll be able to …

• describe vectors in terms of magnitude and direction
• define scalar
• add, subtract, (element-wise) multiply, and (element-wise) divide vectors
• add, subtract, multiply, and divide vectors using a scalar

Vectors

Vectors can be thought of as n-dimensional points in space.

1D

If we decided to represent words using a single feature (ex. a sentiment score), we could represent them as 1D vectors:

x
0.5
0.75
0.9

A single dimension is pretty limiting, though.

Even if we go beyond a single dimension, we can continue to think of vectors as points in n-dimensional space.

2D

x,y
0,1
12,7
3,3
5,6

3D

x,y,z
0,1,2
12,7,5
3,3,3
5,6,6

n-D

It's quite common to use 100s or 1000s of features (dimensions) to represent words and documents.

Beyond 3 dimensions, though, vectors become less straighforward to visualize. Some options include using properties such as color and shape to represent additional dimensions.

Properties of vectors

All vectors have two components:

1. length or magnitude
2. direction

Magnitude

The magnitude of a vector is its distance from the origin (i.e., the vector comprised of all zeros). Though it can be zero in a special case, length is never negative.

For a vector $\mathbf{a}$, you'll often see $\| \mathbf{a} \|$ used to refer to its magnitude. If we're being more precise ¹, we'll use $\| \mathbf{a} \|_{2}$ to refer to the magnitude of $\mathbf{a}$.

How do we calculate length? One familiar way of thinking about this is to consider a 2D vector and apply the distance formula to it and the origin. For this example, we'll consider the 2D vector $\mathbf{b} = \begin{bmatrix} 4 & 7 \end{bmatrix}$:

$$\begin{aligned} d &= \sqrt{(4 - 0)^{2} + (7 - 0)^{2}} \\[2em] &= \sqrt{16 + 49} \\[2em] &= \sqrt{65} \\[2em] &\approx 8.06 \\[2em] \|\mathbf{b} \|_{2} &\approx 8.06 \end{aligned}$$

A 3-dimensional point (vector) will work the same way as will any $n$-dimensional vector.

Norm

To calculate the magnitude (aka 2-norm) of any vector, we can use the following form of the distance formula:

$$\| \mathbf{x} \|_{2} = \sqrt{\sum_{i=0}^{\vert \mathbf{x} \vert} x_{i}^{2}}$$

Unit vector

Mathematical symbols: $\vert \mathbf{x} \vert$

Mathematical symbols: $\Sigma$

Direction

The other component of a vector is its direction.

To calculate direction, we need to normalize a vector $\mathbf{x}$ by $\|\mathbf{x}\|_{2}$. We do this by dividing each element $x_{i}$ of $\mathbf{x}$ by $\|\mathbf{x}\|$. As an example, consider again the 2D vector $\mathbf{b} = \begin{bmatrix} 4 & 7 \end{bmatrix}$:

$$\begin{aligned} \mathbf{b} &= \begin{bmatrix} 4 & 7 \\ \end{bmatrix} \\[2em] \|\mathbf{b} \|_{2} &\approx 8.06 \\[2em] \frac{\mathbf{b}}{\|\mathbf{b} \|_{2}} &= \begin{bmatrix} \frac{4}{\|\mathbf{b} \|_{2}} & \frac{7}{\|\mathbf{b} \|_{2}} \\ \end{bmatrix} \\[2em] \text{norm}(\mathbf{b}) &\approx \begin{bmatrix} 0.496 & 0.868 \\ \end{bmatrix} \end{aligned}$$

This resulting vector is known as a unit vector. It contains information solely about the direction of the vector.

We can can simplify the formula for normalizing a vector $\mathbf{x}$ as …

$$\text{norm}(\mathbf{x}) = \frac{\mathbf{x}}{\| \mathbf{x}\|_{2}}$$

Operations

We can perform operations such as addition, subtraction, multiplication², and division on a vector and a scalar or a vector and another vector.

Scalar and vector

What is a scalar?

A scalar is a number that scales a vector.

Say we have a two dimensional vector, $\mathbf{b}$:

$$\mathbf{b} = \begin{bmatrix} 2 & 3 \\ \end{bmatrix}$$

We can visualize it as a single point:

If multiply this vector by the scalar 2, we get the following result:

$$\begin{aligned} 2 \begin{bmatrix} 2 & 3 \\ \end{bmatrix} &= \begin{bmatrix} 2 \times 2 & 2 \times 3 \\ \end{bmatrix} \\[2em] &= \begin{bmatrix} 4 & 6 \\ \end{bmatrix} \end{aligned}$$

Vector and vector

Vector-vector operations rely on the two vectors having the same shape (dimensionality). These operations are performed against pairs of elements in the two vectors that share the same position (index). Let's look at a couple of examples …

Subtraction

$$\begin{bmatrix} a \\ b \\ c \\ \end{bmatrix} - \begin{bmatrix} d \\ e \\ f \\ \end{bmatrix} = \begin{bmatrix} a - d \\ b - e \\ c - f \\ \end{bmatrix}$$

Using NumPy

Invoke the IPython interpreter via docker:

[hahnpowell@gubuntu] $docker run -it "parsertongue/python:latest" ipython

# run using the following command:
# docker run -it "parsertongue/python:latest" ipython

import numpy as np

x = np.array([1, 2, 4])
y = np.array([3, 4, 3])

x - y

The result:

np.array([-2, -2, 1])

Element-wise Multiplication

$$\begin{bmatrix} a \\ b \\ c \\ \end{bmatrix} \odot \begin{bmatrix} d \\ e \\ f \\ \end{bmatrix} = \begin{bmatrix} a \times d \\ b \times e \\ c \times f \\ \end{bmatrix}$$

Using NumPy

Invoke the IPython interpreter via docker:

[hahnpowell@gubuntu] $docker run -it "parsertongue/python:latest" ipython

# run using the following command:
# docker run -it "parsertongue/python:latest" ipython

import numpy as np

x = np.array([1, 2, 4])
y = np.array([3, 4, 3])
# when vectors have the same shape,
# the * operator performs element-wise
# multiplication
x * y

The result:

np.array([3, 8, 12])

Next steps

You now know some basic vector operations. Before moving on to learning about comparing vectors, let's practice …

Practice

• Given $\mathbf{a} = \begin{bmatrix} 12 & 30 \end{bmatrix}$, what is its magnitude?

• Given $\mathbf{a} = \begin{bmatrix} 2 & 3 \end{bmatrix}$, what vector has the same direction, but 3 times the magnitude of $\mathbf{a}$?

• Given $\mathbf{a} = \begin{bmatrix} 2 \end{bmatrix}$, what vector has the same magnitude, but the opposite direction to $\mathbf{a}$?

Footnotes