diff --git a/tutorials/tutorial4_root_finding.ipynb b/tutorials/tutorial4_root_finding.ipynb new file mode 100644 index 0000000..33c9d07 --- /dev/null +++ b/tutorials/tutorial4_root_finding.ipynb @@ -0,0 +1,243 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "id": "3c51341a", + "metadata": {}, + "source": [ + "# Tutorial 4: Root-Finding Methods\n", + "---\n", + "In this tutorial, we study classical **root-finding algorithms** for nonlinear equations. We will:\n", + "\n", + "- Define the root-finding problem mathematically\n", + "- Derive several algorithms (bisection, fixed-point, Newton, secant)\n", + "- Discuss convergence conditions and error behavior\n", + "- Compare methods with worked examples using the `numethods` package.\n" + ] + }, + { + "cell_type": "markdown", + "id": "2999aa4f", + "metadata": {}, + "source": [ + "## 1. Problem Setup and Notation\n", + "\n", + "We seek to solve a nonlinear scalar equation:\n", + "$$\n", + "f(x) = 0, \\quad f: \\mathbb{R} \\to \\mathbb{R}.\n", + "$$\n", + "with a continuously differentiable function.\n", + "\n", + "\n", + "### Root, residual, and error\n", + "- A **root** $r$ satisfies $f(r)=0$.\n", + "- **Absolute error:** $(e_k = |x_k - x^\\star|)$.\n", + "- **Residual:** $(r_k = |f(x_k)|)$. \n", + "Note that small residual does not always imply small error.\n", + "\n", + "### Multiplicity\n", + "A root $r$ has **multiplicity** $m$ if\n", + "$$\n", + "f(r) = f'(r) = \\dots = f^{(m-1)}(r) = 0, \\quad f^{(m)}(r) \\neq 0.\n", + "$$\n", + "If $x^\\star$ satisfies $f(x^\\star)=0$ and $f'(x^\\star)\\ne 0$, we say the root is **simple** (multiplicity 1).\n", + "\n", + "If $f'(x^\\star)=\\cdots=f^{(m-1)}(x^\\star)=0$ and $f^{(m)}(x^\\star)\\ne 0$, we say the root has **multiplicity** (m).\n", + "\n", + "- **Simple roots** ($m=1$): most methods converge rapidly.\n", + "- **Multiple roots** ($m>1$): convergence often slows.\n" + ] + }, + { + "cell_type": "markdown", + "id": "b610cea2", + "metadata": {}, + "source": [ + "## 2. Bisection Method\n", + "\n", + "**Assumption (Intermediate Value Theorem):** If f is continuous on ([a,b]) and (f(a),f(b) < 0),\n", + "then there exists $x^\\star$ in (a,b) with $f(x^\\star)=0$.\n", + "\n", + "- Assumes $f$ is continuous on $[a,b]$ with $f(a)f(b)<0$.\n", + "- Repeatedly bisect interval and select subinterval containing the root.\n", + "\n", + "**Iteration:**\n", + "$$\n", + "c_k = \\frac{a_k+b_k}{2}, \\quad f(c_k).\n", + "$$\n", + "\n", + "**Error bound:** interval length halves each step:\n", + "$$\n", + "|c_k-r| \\le \\frac{b-a}{2^k}.\n", + "$$\n", + "- Convergence: **linear**, guaranteed.\n" + ] + }, + { + "cell_type": "markdown", + "id": "3be4a510", + "metadata": {}, + "source": [ + "## 3. Fixed-Point Iteration\n", + "- Rewrite equation as $x=g(x)$.\n", + "- Iterate:\n", + "$$\n", + "x_{k+1} = g(x_k).\n", + "$$\n", + "\n", + "**Convergence theorem (Banach fixed-point):** If g is continuously differentiable near $(x^\\star)$ and\n", + "$$\n", + "|g'(x_\\star)| < 1,\n", + "$$\n", + "then for initial guesses $x_0$ sufficiently close to $x^\\star$, the iterates converge **linearly** to $x^\\star$ with asymptotic rate $|g'(x^\\star)|$.\n", + "\n", + "**Choice of g.** Different rearrangements yield different g's with different convergence properties.\n", + "A poor choice (with $(|g'|\\ge 1))$ can diverge.\n", + "\n", + "- Rate: linear with factor $|g'(r)|$.\n" + ] + }, + { + "cell_type": "markdown", + "id": "40e66a30", + "metadata": {}, + "source": [ + "## 4. Newton’s Method\n", + "From Taylor expansion:\n", + "$$\n", + "f(x) \\approx f(x_k) + f'(x_k)(x-x_k).\n", + "$$\n", + "Set $f(x)=0$ to solve for next iterate:\n", + "$$\n", + "x_{k+1} = x_k - \\frac{f(x_k)}{f'(x_k)}.\n", + "$$\n", + "\n", + "- **Quadratic convergence** for simple roots.\n", + "- For multiple roots: drops to linear.\n", + "- Requires derivative, sensitive to initial guess.\n" + ] + }, + { + "cell_type": "markdown", + "id": "84888305", + "metadata": {}, + "source": [ + "## 5. Secant Method\n", + "- Avoids derivative by approximating slope with finite difference:\n", + "$$\n", + "x_{k+1} = x_k - f(x_k) \\frac{x_k - x_{k-1}}{f(x_k)-f(x_{k-1})}.\n", + "$$\n", + "\n", + "- Convergence order: $\\approx 1.618$ (superlinear).\n", + "- More efficient than Newton if derivative expensive.\n" + ] + }, + { + "cell_type": "markdown", + "id": "7ed8a0b5", + "metadata": {}, + "source": [ + "## 6. Stopping Criteria\n", + "We stop iteration when:\n", + "- $|f(x_k)| \\le \\varepsilon$ (residual small), or\n", + "- $|x_{k+1}-x_k| \\le \\varepsilon(1+|x_{k+1}|)$ (relative step small).\n" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "id": "4c9f93d9", + "metadata": {}, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Bisection root: 1.4142135605216026\n", + "Newton root: 1.4142135623730951\n", + "Secant root: 1.414213562373095\n", + "Fixed-point failed: Fixed-point iteration did not converge\n" + ] + } + ], + "source": [ + "from numethods import Bisection, FixedPoint, NewtonRoot, Secant\n", + "import math\n", + "\n", + "# Example function: f(x) = x^2 - 2\n", + "f = lambda x: x**2 - 2\n", + "df = lambda x: 2*x\n", + "\n", + "# Bisection\n", + "bisect = Bisection(f, 0, 2, tol=1e-8)\n", + "root_b = bisect.solve()\n", + "print('Bisection root:', root_b)\n", + "\n", + "# Newton\n", + "newton = NewtonRoot(f, df, 1.0, tol=1e-12)\n", + "root_n = newton.solve()\n", + "print('Newton root:', root_n)\n", + "\n", + "# Secant\n", + "sec = Secant(f, 0, 2, tol=1e-12)\n", + "root_s = sec.solve()\n", + "print('Secant root:', root_s)\n", + "\n", + "# Fixed point: g(x)=sqrt(2) ~ rewriting\n", + "g = lambda x: (2/x) # not always convergent, but demonstrates\n", + "try:\n", + " fp = FixedPoint(g, 1.0, tol=1e-8)\n", + " root_fp = fp.solve()\n", + " print('Fixed-point root:', root_fp)\n", + "except Exception as e:\n", + " print('Fixed-point failed:', e)\n" + ] + }, + { + "cell_type": "markdown", + "id": "833b538c", + "metadata": {}, + "source": [ + "## 7. Comparison of Methods\n", + "| Method | Requires derivative | Convergence rate | Guarantee? |\n", + "|--------|---------------------|------------------|------------|\n", + "| Bisection | No | Linear | Yes (if sign change) |\n", + "| Fixed-Point | No | Linear | Not always |\n", + "| Newton | Yes | Quadratic | Locally (good guess) |\n", + "| Secant | No | ~1.618 (superlinear) | Locally (good guess) |\n" + ] + }, + { + "cell_type": "markdown", + "id": "0d372aa0", + "metadata": {}, + "source": [ + "## 8. Exercises\n", + "1. Apply all four methods to $f(x)=\\cos x - x$.\n", + "2. Try Newton’s method on $f(x)=(x-1)^2$ and compare convergence rate with $f(x)=x^2-2$.\n", + "3. Modify Secant to stop if denominator too small.\n" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "Python 3", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.13.7" + } + }, + "nbformat": 4, + "nbformat_minor": 5 +}