{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3c51341a",
   "metadata": {},
   "source": [
    "# Tutorial 4: Root-Finding Methods\n",
    "---\n",
    "In this tutorial, we study classical **root-finding algorithms** for nonlinear equations. We will:\n",
    "\n",
    "- Define the root-finding problem mathematically\n",
    "- Derive several algorithms (bisection, fixed-point, Newton, secant)\n",
    "- Discuss convergence conditions and error behavior\n",
    "- Compare methods with worked examples using the `numethods` package.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2999aa4f",
   "metadata": {},
   "source": [
    "## 1. Problem Setup and Notation\n",
    "\n",
    "We seek to solve a nonlinear scalar equation:\n",
    "$$\n",
    "f(x) = 0, \\quad f: \\mathbb{R} \\to \\mathbb{R}.\n",
    "$$\n",
    "with a continuously differentiable function.\n",
    "\n",
    "\n",
    "### Root, residual, and error\n",
    "- A **root** $r$ satisfies $f(r)=0$.\n",
    "- **Absolute error:** $(e_k = |x_k - x^\\star|)$.\n",
    "- **Residual:** $(r_k = |f(x_k)|)$. \n",
    "Note that small residual does not always imply small error.\n",
    "\n",
    "### Multiplicity\n",
    "A root $r$ has **multiplicity** $m$ if\n",
    "$$\n",
    "f(r) = f'(r) = \\dots = f^{(m-1)}(r) = 0, \\quad f^{(m)}(r) \\neq 0.\n",
    "$$\n",
    "If $x^\\star$ satisfies $f(x^\\star)=0$ and $f'(x^\\star)\\ne 0$, we say the root is **simple** (multiplicity 1).\n",
    "\n",
    "If $f'(x^\\star)=\\cdots=f^{(m-1)}(x^\\star)=0$ and $f^{(m)}(x^\\star)\\ne 0$, we say the root has **multiplicity** (m).\n",
    "\n",
    "- **Simple roots** ($m=1$): most methods converge rapidly.\n",
    "- **Multiple roots** ($m>1$): convergence often slows.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b610cea2",
   "metadata": {},
   "source": [
    "## 2. Bisection Method\n",
    "\n",
    "**Assumption (Intermediate Value Theorem):** If f is continuous on ([a,b]) and (f(a),f(b) < 0),\n",
    "then there exists $x^\\star$ in (a,b) with $f(x^\\star)=0$.\n",
    "\n",
    "- Assumes $f$ is continuous on $[a,b]$ with $f(a)f(b)<0$.\n",
    "- Repeatedly bisect interval and select subinterval containing the root.\n",
    "\n",
    "**Iteration:**\n",
    "$$\n",
    "c_k = \\frac{a_k+b_k}{2}, \\quad f(c_k).\n",
    "$$\n",
    "\n",
    "**Error bound:** interval length halves each step:\n",
    "$$\n",
    "|c_k-r| \\le \\frac{b-a}{2^k}.\n",
    "$$\n",
    "- Convergence: **linear**, guaranteed.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3be4a510",
   "metadata": {},
   "source": [
    "## 3. Fixed-Point Iteration\n",
    "- Rewrite equation as $x=g(x)$.\n",
    "- Iterate:\n",
    "$$\n",
    "x_{k+1} = g(x_k).\n",
    "$$\n",
    "\n",
    "**Convergence theorem (Banach fixed-point):** If g is continuously differentiable near $(x^\\star)$ and\n",
    "$$\n",
    "|g'(x_\\star)| < 1,\n",
    "$$\n",
    "then for initial guesses $x_0$ sufficiently close to $x^\\star$, the iterates converge **linearly** to $x^\\star$ with asymptotic rate $|g'(x^\\star)|$.\n",
    "\n",
    "**Choice of g.** Different rearrangements yield different g's with different convergence properties.\n",
    "A poor choice (with $(|g'|\\ge 1))$ can diverge.\n",
    "\n",
    "- Rate: linear with factor $|g'(r)|$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "40e66a30",
   "metadata": {},
   "source": [
    "## 4. Newton’s Method\n",
    "From Taylor expansion:\n",
    "$$\n",
    "f(x) \\approx f(x_k) + f'(x_k)(x-x_k).\n",
    "$$\n",
    "Set $f(x)=0$ to solve for next iterate:\n",
    "$$\n",
    "x_{k+1} = x_k - \\frac{f(x_k)}{f'(x_k)}.\n",
    "$$\n",
    "\n",
    "- **Quadratic convergence** for simple roots.\n",
    "- For multiple roots: drops to linear.\n",
    "- Requires derivative, sensitive to initial guess.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84888305",
   "metadata": {},
   "source": [
    "## 5. Secant Method\n",
    "- Avoids derivative by approximating slope with finite difference:\n",
    "$$\n",
    "x_{k+1} = x_k - f(x_k) \\frac{x_k - x_{k-1}}{f(x_k)-f(x_{k-1})}.\n",
    "$$\n",
    "\n",
    "- Convergence order: $\\approx 1.618$ (superlinear).\n",
    "- More efficient than Newton if derivative expensive.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ed8a0b5",
   "metadata": {},
   "source": [
    "## 6. Stopping Criteria\n",
    "We stop iteration when:\n",
    "- $|f(x_k)| \\le \\varepsilon$ (residual small), or\n",
    "- $|x_{k+1}-x_k| \\le \\varepsilon(1+|x_{k+1}|)$ (relative step small).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4c9f93d9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Bisection root: 1.4142135605216026\n",
      "Newton root: 1.4142135623730951\n",
      "Secant root: 1.414213562373095\n",
      "Fixed-point failed: Fixed-point iteration did not converge\n"
     ]
    }
   ],
   "source": [
    "from numethods import Bisection, FixedPoint, NewtonRoot, Secant\n",
    "import math\n",
    "\n",
    "# Example function: f(x) = x^2 - 2\n",
    "f = lambda x: x**2 - 2\n",
    "df = lambda x: 2*x\n",
    "\n",
    "# Bisection\n",
    "bisect = Bisection(f, 0, 2, tol=1e-8)\n",
    "root_b = bisect.solve()\n",
    "print('Bisection root:', root_b)\n",
    "\n",
    "# Newton\n",
    "newton = NewtonRoot(f, df, 1.0, tol=1e-12)\n",
    "root_n = newton.solve()\n",
    "print('Newton root:', root_n)\n",
    "\n",
    "# Secant\n",
    "sec = Secant(f, 0, 2, tol=1e-12)\n",
    "root_s = sec.solve()\n",
    "print('Secant root:', root_s)\n",
    "\n",
    "# Fixed point: g(x)=sqrt(2) ~ rewriting\n",
    "g = lambda x: (2/x)  # not always convergent, but demonstrates\n",
    "try:\n",
    "    fp = FixedPoint(g, 1.0, tol=1e-8)\n",
    "    root_fp = fp.solve()\n",
    "    print('Fixed-point root:', root_fp)\n",
    "except Exception as e:\n",
    "    print('Fixed-point failed:', e)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "833b538c",
   "metadata": {},
   "source": [
    "## 7. Comparison of Methods\n",
    "| Method | Requires derivative | Convergence rate | Guarantee? |\n",
    "|--------|---------------------|------------------|------------|\n",
    "| Bisection | No | Linear | Yes (if sign change) |\n",
    "| Fixed-Point | No | Linear | Not always |\n",
    "| Newton | Yes | Quadratic | Locally (good guess) |\n",
    "| Secant | No | ~1.618 (superlinear) | Locally (good guess) |\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0d372aa0",
   "metadata": {},
   "source": [
    "## 8. Exercises\n",
    "1. Apply all four methods to $f(x)=\\cos x - x$.\n",
    "2. Try Newton’s method on $f(x)=(x-1)^2$ and compare convergence rate with $f(x)=x^2-2$.\n",
    "3. Modify Secant to stop if denominator too small.\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}