forked from denizdonmez/numethods
Upload files to "tutorials"
This commit is contained in:
243
tutorials/tutorial4_root_finding.ipynb
Normal file
243
tutorials/tutorial4_root_finding.ipynb
Normal file
@@ -0,0 +1,243 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3c51341a",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Tutorial 4: Root-Finding Methods\n",
|
||||
"---\n",
|
||||
"In this tutorial, we study classical **root-finding algorithms** for nonlinear equations. We will:\n",
|
||||
"\n",
|
||||
"- Define the root-finding problem mathematically\n",
|
||||
"- Derive several algorithms (bisection, fixed-point, Newton, secant)\n",
|
||||
"- Discuss convergence conditions and error behavior\n",
|
||||
"- Compare methods with worked examples using the `numethods` package.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "2999aa4f",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 1. Problem Setup and Notation\n",
|
||||
"\n",
|
||||
"We seek to solve a nonlinear scalar equation:\n",
|
||||
"$$\n",
|
||||
"f(x) = 0, \\quad f: \\mathbb{R} \\to \\mathbb{R}.\n",
|
||||
"$$\n",
|
||||
"with a continuously differentiable function.\n",
|
||||
"\n",
|
||||
"\n",
|
||||
"### Root, residual, and error\n",
|
||||
"- A **root** $r$ satisfies $f(r)=0$.\n",
|
||||
"- **Absolute error:** $(e_k = |x_k - x^\\star|)$.\n",
|
||||
"- **Residual:** $(r_k = |f(x_k)|)$. \n",
|
||||
"Note that small residual does not always imply small error.\n",
|
||||
"\n",
|
||||
"### Multiplicity\n",
|
||||
"A root $r$ has **multiplicity** $m$ if\n",
|
||||
"$$\n",
|
||||
"f(r) = f'(r) = \\dots = f^{(m-1)}(r) = 0, \\quad f^{(m)}(r) \\neq 0.\n",
|
||||
"$$\n",
|
||||
"If $x^\\star$ satisfies $f(x^\\star)=0$ and $f'(x^\\star)\\ne 0$, we say the root is **simple** (multiplicity 1).\n",
|
||||
"\n",
|
||||
"If $f'(x^\\star)=\\cdots=f^{(m-1)}(x^\\star)=0$ and $f^{(m)}(x^\\star)\\ne 0$, we say the root has **multiplicity** (m).\n",
|
||||
"\n",
|
||||
"- **Simple roots** ($m=1$): most methods converge rapidly.\n",
|
||||
"- **Multiple roots** ($m>1$): convergence often slows.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "b610cea2",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 2. Bisection Method\n",
|
||||
"\n",
|
||||
"**Assumption (Intermediate Value Theorem):** If f is continuous on ([a,b]) and (f(a),f(b) < 0),\n",
|
||||
"then there exists $x^\\star$ in (a,b) with $f(x^\\star)=0$.\n",
|
||||
"\n",
|
||||
"- Assumes $f$ is continuous on $[a,b]$ with $f(a)f(b)<0$.\n",
|
||||
"- Repeatedly bisect interval and select subinterval containing the root.\n",
|
||||
"\n",
|
||||
"**Iteration:**\n",
|
||||
"$$\n",
|
||||
"c_k = \\frac{a_k+b_k}{2}, \\quad f(c_k).\n",
|
||||
"$$\n",
|
||||
"\n",
|
||||
"**Error bound:** interval length halves each step:\n",
|
||||
"$$\n",
|
||||
"|c_k-r| \\le \\frac{b-a}{2^k}.\n",
|
||||
"$$\n",
|
||||
"- Convergence: **linear**, guaranteed.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "3be4a510",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 3. Fixed-Point Iteration\n",
|
||||
"- Rewrite equation as $x=g(x)$.\n",
|
||||
"- Iterate:\n",
|
||||
"$$\n",
|
||||
"x_{k+1} = g(x_k).\n",
|
||||
"$$\n",
|
||||
"\n",
|
||||
"**Convergence theorem (Banach fixed-point):** If g is continuously differentiable near $(x^\\star)$ and\n",
|
||||
"$$\n",
|
||||
"|g'(x_\\star)| < 1,\n",
|
||||
"$$\n",
|
||||
"then for initial guesses $x_0$ sufficiently close to $x^\\star$, the iterates converge **linearly** to $x^\\star$ with asymptotic rate $|g'(x^\\star)|$.\n",
|
||||
"\n",
|
||||
"**Choice of g.** Different rearrangements yield different g's with different convergence properties.\n",
|
||||
"A poor choice (with $(|g'|\\ge 1))$ can diverge.\n",
|
||||
"\n",
|
||||
"- Rate: linear with factor $|g'(r)|$.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "40e66a30",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 4. Newton’s Method\n",
|
||||
"From Taylor expansion:\n",
|
||||
"$$\n",
|
||||
"f(x) \\approx f(x_k) + f'(x_k)(x-x_k).\n",
|
||||
"$$\n",
|
||||
"Set $f(x)=0$ to solve for next iterate:\n",
|
||||
"$$\n",
|
||||
"x_{k+1} = x_k - \\frac{f(x_k)}{f'(x_k)}.\n",
|
||||
"$$\n",
|
||||
"\n",
|
||||
"- **Quadratic convergence** for simple roots.\n",
|
||||
"- For multiple roots: drops to linear.\n",
|
||||
"- Requires derivative, sensitive to initial guess.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "84888305",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 5. Secant Method\n",
|
||||
"- Avoids derivative by approximating slope with finite difference:\n",
|
||||
"$$\n",
|
||||
"x_{k+1} = x_k - f(x_k) \\frac{x_k - x_{k-1}}{f(x_k)-f(x_{k-1})}.\n",
|
||||
"$$\n",
|
||||
"\n",
|
||||
"- Convergence order: $\\approx 1.618$ (superlinear).\n",
|
||||
"- More efficient than Newton if derivative expensive.\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "7ed8a0b5",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 6. Stopping Criteria\n",
|
||||
"We stop iteration when:\n",
|
||||
"- $|f(x_k)| \\le \\varepsilon$ (residual small), or\n",
|
||||
"- $|x_{k+1}-x_k| \\le \\varepsilon(1+|x_{k+1}|)$ (relative step small).\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 1,
|
||||
"id": "4c9f93d9",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Bisection root: 1.4142135605216026\n",
|
||||
"Newton root: 1.4142135623730951\n",
|
||||
"Secant root: 1.414213562373095\n",
|
||||
"Fixed-point failed: Fixed-point iteration did not converge\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from numethods import Bisection, FixedPoint, NewtonRoot, Secant\n",
|
||||
"import math\n",
|
||||
"\n",
|
||||
"# Example function: f(x) = x^2 - 2\n",
|
||||
"f = lambda x: x**2 - 2\n",
|
||||
"df = lambda x: 2*x\n",
|
||||
"\n",
|
||||
"# Bisection\n",
|
||||
"bisect = Bisection(f, 0, 2, tol=1e-8)\n",
|
||||
"root_b = bisect.solve()\n",
|
||||
"print('Bisection root:', root_b)\n",
|
||||
"\n",
|
||||
"# Newton\n",
|
||||
"newton = NewtonRoot(f, df, 1.0, tol=1e-12)\n",
|
||||
"root_n = newton.solve()\n",
|
||||
"print('Newton root:', root_n)\n",
|
||||
"\n",
|
||||
"# Secant\n",
|
||||
"sec = Secant(f, 0, 2, tol=1e-12)\n",
|
||||
"root_s = sec.solve()\n",
|
||||
"print('Secant root:', root_s)\n",
|
||||
"\n",
|
||||
"# Fixed point: g(x)=sqrt(2) ~ rewriting\n",
|
||||
"g = lambda x: (2/x) # not always convergent, but demonstrates\n",
|
||||
"try:\n",
|
||||
" fp = FixedPoint(g, 1.0, tol=1e-8)\n",
|
||||
" root_fp = fp.solve()\n",
|
||||
" print('Fixed-point root:', root_fp)\n",
|
||||
"except Exception as e:\n",
|
||||
" print('Fixed-point failed:', e)\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "833b538c",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 7. Comparison of Methods\n",
|
||||
"| Method | Requires derivative | Convergence rate | Guarantee? |\n",
|
||||
"|--------|---------------------|------------------|------------|\n",
|
||||
"| Bisection | No | Linear | Yes (if sign change) |\n",
|
||||
"| Fixed-Point | No | Linear | Not always |\n",
|
||||
"| Newton | Yes | Quadratic | Locally (good guess) |\n",
|
||||
"| Secant | No | ~1.618 (superlinear) | Locally (good guess) |\n"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "0d372aa0",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## 8. Exercises\n",
|
||||
"1. Apply all four methods to $f(x)=\\cos x - x$.\n",
|
||||
"2. Try Newton’s method on $f(x)=(x-1)^2$ and compare convergence rate with $f(x)=x^2-2$.\n",
|
||||
"3. Modify Secant to stop if denominator too small.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": "Python 3",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.13.7"
|
||||
}
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
Reference in New Issue
Block a user