numethods/tutorials/tutorial1_vectors_matrices.ipynb

Tutorial 1 - Vectors & Matrices in numethods

---

1. Introduction

In numerical computing, vectors and matrices are the basic objects.
Almost every algorithm (linear solvers, eigenvalue problems, optimization, curve fitting, etc.) is built upon them.

In this tutorial, we will:

• Define what vectors and matrices are.
• Review their basic operations (addition, scalar multiplication, multiplication).
• Define and compute vector and matrix norms.
• Show how these operations work in the numethods package.

2. Importing our package

from numethods.linalg import Vector, Matrix

3. What is a vector?

A vector is an ordered collection of numbers (scalars).
We can think of a vector as a column:

\[ v = \begin{bmatrix} v_1 \\ v_2 \\ \vdots \\ v_n \end{bmatrix}, \quad v \in \mathbb{R}^n \]

or as a row:

\[ v^T = \begin{bmatrix} v_1 & v_2 & \cdots & v_n \end{bmatrix}. \]

Example

A vector in \(\mathbb{R}^3\):

\[ v = \begin{bmatrix} 1 \\ 2 \\ 3 \end{bmatrix}. \]

v = Vector([1, 2, 3])
print("v =", v)

v = Vector([1.0, 2.0, 3.0])

4. What is a matrix?

A matrix is a rectangular array of numbers with rows and columns:

\[ A = \begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn} \end{bmatrix}, \quad A \in \mathbb{R}^{m \times n} \]

A = Matrix([[1, 2, 3],
            [4, 5, 6],
            [7, 8, 9]])
print("A =\n", A)

A =
 Matrix([[1.0, 2.0, 3.0], [4.0, 5.0, 6.0], [7.0, 8.0, 9.0]])

5. Basic operations

5.1 Vector addition and subtraction

For \(u, v \in \mathbb{R}^n\):

\[ u + v = \begin{bmatrix} u_1 + v_1 \\ u_2 + v_2 \\ \vdots \\ u_n + v_n \end{bmatrix}, \quad u - v = \begin{bmatrix} u_1 - v_1 \\ u_2 - v_2 \\ \vdots \\ u_n - v_n \end{bmatrix}. \]

u = Vector([3, 2, 1])
v = Vector([1, 2, 3])

print("u + v =", u + v)
print("u - v =", u - v)

u + v = Vector([4.0, 4.0, 4.0])
u - v = Vector([2.0, 0.0, -2.0])

5.2 Scalar multiplication

For \(\alpha \in \mathbb{R}, v \in \mathbb{R}^n\):

\[ \alpha v = \begin{bmatrix} \alpha v_1 \\ \alpha v_2 \\ \vdots \\ \alpha v_n \end{bmatrix}. \]

v = Vector([1, 2, 3])

print("2 * v =", 2 * v)
print("v * 2 =", v * 2)

2 * v = Vector([2.0, 4.0, 6.0])
v * 2 = Vector([2.0, 4.0, 6.0])

5.3 Matrix addition and subtraction

For \(A, B \in \mathbb{R}^{m \times n}\):

\[ A + B = [ a_{ij} + b_{ij} ], \quad A - B = [ a_{ij} - b_{ij} ]. \]

A = Matrix([[1, 2], [3, 4]])
B = Matrix([[5, 6], [7, 8]])

print("A + B =\n", A + B)
print("A - B =\n", A - B)

A + B =
 Matrix([[6.0, 8.0], [10.0, 12.0]])
A - B =
 Matrix([[-4.0, -4.0], [-4.0, -4.0]])

5.4 Matrix-Vector multiplication

For \(A \in \mathbb{R}^{m \times n}, v \in \mathbb{R}^n\):

\[ (Av)_i = \sum_{j=1}^n a_{ij} v_j. \]

A = Matrix([[1, 2, 3],
            [4, 5, 6],
            [7, 8, 9]])
v = Vector([1, 0, -1])

print("A @ v =", A @ v)

A @ v = Vector([-2.0, -2.0, -2.0])

5.5 Matrix-Matrix multiplication

For \(A \in \mathbb{R}^{m \times n}, B \in \mathbb{R}^{n \times p}\):

\[ (AB)_{ij} = \sum_{k=1}^n a_{ik} b_{kj}. \]

A = Matrix([[1, 2],
            [3, 4]])
B = Matrix([[2, 0],
            [1, 2]])

print("A @ B =\n", A @ B)

A @ B =
 Matrix([[4.0, 4.0], [10.0, 8.0]])

5.6 Transpose

For \(A \in \mathbb{R}^{m \times n}\):

\[ A^T_{ij} = A_{ji}. \]

A = Matrix([[1, 2, 3],
            [4, 5, 6]])

print("A^T =\n", A.T)

A^T =
 Matrix([[1.0, 4.0], [2.0, 5.0], [3.0, 6.0]])

6. Norms

Norms measure the size or length of vectors and matrices.

6.1 Vector norms

• 1-norm: \(\|v\|_1 = \sum |v_i|\)
  
• 2-norm (Euclidean): \(\|v\|_2 = \sqrt{\sum v_i^2}\)
  
• ∞-norm: \(\|v\|_\infty = \max |v_i|\)

v = Vector([3, -4, 5])

print("‖v‖₁ =", v.norm())
print("‖v‖₂ =", v.norm2())
print("‖v‖∞ =", v.norm_inf())

‖v‖₁ = 12.0
‖v‖₂ = 7.0710678118654755
‖v‖∞ = 5.0

6.2 Matrix norms

• Frobenius norm: \(\|A\|_F = \sqrt{\sum_{i,j} a_{ij}^2}\)
  
• 1-norm (maximum column sum): \(\|A\|_1 = \max_j \sum_i |a_{ij}|\)
  
• ∞-norm (maximum row sum): \(\|A\|_\infty = \max_i \sum_j |a_{ij}|\)

A = Matrix([[1, -2],
            [3,  4]])

print("‖A‖_F =", A.norm_fro())
print("‖A‖₁ =", A.norm())
print("‖A‖∞ =", A.norm_inf())

‖A‖_F = 5.477225575051661
‖A‖₁ = 6.0
‖A‖∞ = 7.0

7. Summary

• A vector is an element of \(\mathbb{R}^n\), a list of numbers.
  
• A matrix is a rectangular array of numbers \(\mathbb{R}^{m \times n}\).
  
• We can add, subtract, and scale vectors/matrices.
  
• Multiplication extends naturally: matrix-vector and matrix-matrix.
  
• Transpose flips rows and columns.
  
• Norms measure the size of vectors and matrices.