update tutorial4_root_finding.ipynb to add trace printing for all methods and handle exceptions in FixedPoint method
This commit is contained in:
Deniz
@@ -73,80 +73,10 @@
|
|||||||
"- Convergence: **linear**, guaranteed.\n"
|
"- Convergence: **linear**, guaranteed.\n"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"id": "3be4a510",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## 3. Fixed-Point Iteration\n",
|
|
||||||
"- Rewrite equation as $x=g(x)$.\n",
|
|
||||||
"- Iterate:\n",
|
|
||||||
"$$\n",
|
|
||||||
"x_{k+1} = g(x_k).\n",
|
|
||||||
"$$\n",
|
|
||||||
"\n",
|
|
||||||
"**Convergence theorem (Banach fixed-point):** If g is continuously differentiable near $(x^\\star)$ and\n",
|
|
||||||
"$$\n",
|
|
||||||
"|g'(x_\\star)| < 1,\n",
|
|
||||||
"$$\n",
|
|
||||||
"then for initial guesses $x_0$ sufficiently close to $x^\\star$, the iterates converge **linearly** to $x^\\star$ with asymptotic rate $|g'(x^\\star)|$.\n",
|
|
||||||
"\n",
|
|
||||||
"**Choice of g.** Different rearrangements yield different g's with different convergence properties.\n",
|
|
||||||
"A poor choice (with $(|g'|\\ge 1))$ can diverge.\n",
|
|
||||||
"\n",
|
|
||||||
"- Rate: linear with factor $|g'(r)|$.\n"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"id": "40e66a30",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## 4. Newton’s Method\n",
|
|
||||||
"From Taylor expansion:\n",
|
|
||||||
"$$\n",
|
|
||||||
"f(x) \\approx f(x_k) + f'(x_k)(x-x_k).\n",
|
|
||||||
"$$\n",
|
|
||||||
"Set $f(x)=0$ to solve for next iterate:\n",
|
|
||||||
"$$\n",
|
|
||||||
"x_{k+1} = x_k - \\frac{f(x_k)}{f'(x_k)}.\n",
|
|
||||||
"$$\n",
|
|
||||||
"\n",
|
|
||||||
"- **Quadratic convergence** for simple roots.\n",
|
|
||||||
"- For multiple roots: drops to linear.\n",
|
|
||||||
"- Requires derivative, sensitive to initial guess.\n"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"id": "84888305",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## 5. Secant Method\n",
|
|
||||||
"- Avoids derivative by approximating slope with finite difference:\n",
|
|
||||||
"$$\n",
|
|
||||||
"x_{k+1} = x_k - f(x_k) \\frac{x_k - x_{k-1}}{f(x_k)-f(x_{k-1})}.\n",
|
|
||||||
"$$\n",
|
|
||||||
"\n",
|
|
||||||
"- Convergence order: $\\approx 1.618$ (superlinear).\n",
|
|
||||||
"- More efficient than Newton if derivative expensive.\n"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "markdown",
|
|
||||||
"id": "7ed8a0b5",
|
|
||||||
"metadata": {},
|
|
||||||
"source": [
|
|
||||||
"## 6. Stopping Criteria\n",
|
|
||||||
"We stop iteration when:\n",
|
|
||||||
"- $|f(x_k)| \\le \\varepsilon$ (residual small), or\n",
|
|
||||||
"- $|x_{k+1}-x_k| \\le \\varepsilon(1+|x_{k+1}|)$ (relative step small).\n"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
{
|
||||||
"cell_type": "code",
|
"cell_type": "code",
|
||||||
"execution_count": 1,
|
"execution_count": null,
|
||||||
"id": "4c9f93d9",
|
"id": "d26f75fb",
|
||||||
"metadata": {},
|
"metadata": {},
|
||||||
"outputs": [
|
"outputs": [
|
||||||
{
|
{
|
||||||
@@ -154,75 +84,6 @@
|
|||||||
"output_type": "stream",
|
"output_type": "stream",
|
||||||
"text": [
|
"text": [
|
||||||
"Bisection root: 1.4142135605216026\n",
|
"Bisection root: 1.4142135605216026\n",
|
||||||
"Newton root: 1.4142135623730951\n",
|
|
||||||
"Secant root: 1.414213562373095\n",
|
|
||||||
"Fixed-point failed: Fixed-point iteration did not converge\n"
|
|
||||||
]
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"source": [
|
|
||||||
"from numethods import Bisection, FixedPoint, NewtonRoot, Secant, print_trace\n",
|
|
||||||
"import math\n",
|
|
||||||
"\n",
|
|
||||||
"# Example function: f(x) = x^2 - 2\n",
|
|
||||||
"f = lambda x: x**2 - 2\n",
|
|
||||||
"df = lambda x: 2*x\n",
|
|
||||||
"\n",
|
|
||||||
"# Bisection\n",
|
|
||||||
"bisect = Bisection(f, 0, 2, tol=1e-8)\n",
|
|
||||||
"root_b = bisect.solve()\n",
|
|
||||||
"print('Bisection root:', root_b)\n",
|
|
||||||
"\n",
|
|
||||||
"# Newton\n",
|
|
||||||
"newton = NewtonRoot(f, df, 1.0, tol=1e-12)\n",
|
|
||||||
"root_n = newton.solve()\n",
|
|
||||||
"print('Newton root:', root_n)\n",
|
|
||||||
"\n",
|
|
||||||
"# Secant\n",
|
|
||||||
"sec = Secant(f, 0, 2, tol=1e-12)\n",
|
|
||||||
"root_s = sec.solve()\n",
|
|
||||||
"print('Secant root:', root_s)\n",
|
|
||||||
"\n",
|
|
||||||
"# Fixed point: g(x)=sqrt(2) ~ rewriting\n",
|
|
||||||
"g = lambda x: (2/x) # not always convergent, but demonstrates\n",
|
|
||||||
"try:\n",
|
|
||||||
" fp = FixedPoint(g, 1.0, tol=1e-8)\n",
|
|
||||||
" root_fp = fp.solve()\n",
|
|
||||||
" print('Fixed-point root:', root_fp)\n",
|
|
||||||
"except Exception as e:\n",
|
|
||||||
" print('Fixed-point failed:', e)\n"
|
|
||||||
]
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"cell_type": "code",
|
|
||||||
"execution_count": 2,
|
|
||||||
"id": "7f8fe13e",
|
|
||||||
"metadata": {},
|
|
||||||
"outputs": [
|
|
||||||
{
|
|
||||||
"name": "stdout",
|
|
||||||
"output_type": "stream",
|
|
||||||
"text": [
|
|
||||||
"Newton Method Trace (x^2 - 2):\n",
|
|
||||||
" iter | x | f(x) | df(x) | x_new | error\n",
|
|
||||||
"------------------------------------------------------------------------------\n",
|
|
||||||
" 0 | 1 | -1 | 2 | 1.5 | 0.5\n",
|
|
||||||
" 1 | 1.5 | 0.25 | 3 | 1.41667 | 0.0833333\n",
|
|
||||||
" 2 | 1.41667 | 0.00694444 | 2.83333 | 1.41422 | 0.00245098\n",
|
|
||||||
" 3 | 1.41422 | 6.0073e-06 | 2.82843 | 1.41421 | 2.1239e-06\n",
|
|
||||||
" 4 | 1.41421 | 4.51061e-12 | 2.82843 | 1.41421 | 1.59472e-12\n",
|
|
||||||
"\n",
|
|
||||||
"Secant Method Trace (x^2 - 2):\n",
|
|
||||||
" iter | x0 | x1 | x2 | f(x0) | f(x1) | error\n",
|
|
||||||
"-------------------------------------------------------------------------------------------\n",
|
|
||||||
" 0 | 0 | 2 | 1 | -2 | 2 | 1\n",
|
|
||||||
" 1 | 2 | 1 | 1.33333 | 2 | -1 | 0.333333\n",
|
|
||||||
" 2 | 1 | 1.33333 | 1.42857 | -1 | -0.222222 | 0.0952381\n",
|
|
||||||
" 3 | 1.33333 | 1.42857 | 1.41379 | -0.222222 | 0.0408163 | 0.0147783\n",
|
|
||||||
" 4 | 1.42857 | 1.41379 | 1.41421 | 0.0408163 | -0.00118906 | 0.000418335\n",
|
|
||||||
" 5 | 1.41379 | 1.41421 | 1.41421 | -0.00118906 | -6.00729e-06 | 2.12421e-06\n",
|
|
||||||
" 6 | 1.41421 | 1.41421 | 1.41421 | -6.00729e-06 | 8.93146e-10 | 3.15775e-10\n",
|
|
||||||
" 7 | 1.41421 | 1.41421 | 1.41421 | 8.93146e-10 | -8.88178e-16 | 2.22045e-16\n",
|
|
||||||
"\n",
|
"\n",
|
||||||
"Bisection Method Trace (x^2 - 2):\n",
|
"Bisection Method Trace (x^2 - 2):\n",
|
||||||
" iter | a | b | c | f(a) | f(b) | f(c) | interval\n",
|
" iter | a | b | c | f(a) | f(b) | f(c) | interval\n",
|
||||||
@@ -254,9 +115,63 @@
|
|||||||
" 24 | 1.41421 | 1.41421 | 1.41421 | -6.84571e-08 | 2.68718e-07 | 1.0013e-07 | 1.19209e-07\n",
|
" 24 | 1.41421 | 1.41421 | 1.41421 | -6.84571e-08 | 2.68718e-07 | 1.0013e-07 | 1.19209e-07\n",
|
||||||
" 25 | 1.41421 | 1.41421 | 1.41421 | -6.84571e-08 | 1.0013e-07 | 1.58366e-08 | 5.96046e-08\n",
|
" 25 | 1.41421 | 1.41421 | 1.41421 | -6.84571e-08 | 1.0013e-07 | 1.58366e-08 | 5.96046e-08\n",
|
||||||
" 26 | 1.41421 | 1.41421 | 1.41421 | -6.84571e-08 | 1.58366e-08 | -2.63102e-08 | 2.98023e-08\n",
|
" 26 | 1.41421 | 1.41421 | 1.41421 | -6.84571e-08 | 1.58366e-08 | -2.63102e-08 | 2.98023e-08\n",
|
||||||
" 27 | 1.41421 | 1.41421 | 1.41421 | -2.63102e-08 | 1.58366e-08 | -5.23681e-09 | 1.49012e-08\n",
|
" 27 | 1.41421 | 1.41421 | 1.41421 | -2.63102e-08 | 1.58366e-08 | -5.23681e-09 | 1.49012e-08\n"
|
||||||
" 28 | 1.41421 | 1.41421 | 1.41421 | -5.23681e-09 | 1.58366e-08 | 5.2999e-09 | 7.45058e-09\n",
|
]
|
||||||
" 29 | 1.41421 | 1.41421 | 1.41421 | -5.23681e-09 | 5.2999e-09 | 3.15445e-11 | 3.72529e-09\n",
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"from numethods import Bisection, FixedPoint, NewtonRoot, Secant, print_trace\n",
|
||||||
|
"import math\n",
|
||||||
|
"\n",
|
||||||
|
"# Example function: f(x) = x^2 - 2\n",
|
||||||
|
"f = lambda x: x**2 - 2\n",
|
||||||
|
"df = lambda x: 2*x\n",
|
||||||
|
"\n",
|
||||||
|
"# Bisection\n",
|
||||||
|
"bisect = Bisection(f, 0, 2, tol=1e-8)\n",
|
||||||
|
"root_b = bisect.solve()\n",
|
||||||
|
"print('Bisection root:', root_b)\n",
|
||||||
|
"\n",
|
||||||
|
"steps = bisect.trace()\n",
|
||||||
|
"print(\"\\nBisection Method Trace (x^2 - 2):\")\n",
|
||||||
|
"print_trace(steps)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "3be4a510",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## 3. Fixed-Point Iteration\n",
|
||||||
|
"- Rewrite equation as $x=g(x)$.\n",
|
||||||
|
"- Iterate:\n",
|
||||||
|
"$$\n",
|
||||||
|
"x_{k+1} = g(x_k).\n",
|
||||||
|
"$$\n",
|
||||||
|
"\n",
|
||||||
|
"**Convergence theorem (Banach fixed-point):** If g is continuously differentiable near $(x^\\star)$ and\n",
|
||||||
|
"$$\n",
|
||||||
|
"|g'(x_\\star)| < 1,\n",
|
||||||
|
"$$\n",
|
||||||
|
"then for initial guesses $x_0$ sufficiently close to $x^\\star$, the iterates converge **linearly** to $x^\\star$ with asymptotic rate $|g'(x^\\star)|$.\n",
|
||||||
|
"\n",
|
||||||
|
"**Choice of g.** Different rearrangements yield different g's with different convergence properties.\n",
|
||||||
|
"A poor choice (with $(|g'|\\ge 1))$ can diverge.\n",
|
||||||
|
"\n",
|
||||||
|
"- Rate: linear with factor $|g'(r)|$.\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 2,
|
||||||
|
"id": "436ce6f6",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Fixed-point failed: Fixed-point iteration did not converge\n",
|
||||||
"\n",
|
"\n",
|
||||||
"Fixed-Point Iteration Trace (x^2 - 2):\n",
|
"Fixed-Point Iteration Trace (x^2 - 2):\n",
|
||||||
" iter | x | x_new | error\n",
|
" iter | x | x_new | error\n",
|
||||||
@@ -270,28 +185,137 @@
|
|||||||
}
|
}
|
||||||
],
|
],
|
||||||
"source": [
|
"source": [
|
||||||
"# Newton\n",
|
"# Fixed point: g(x)=sqrt(2) ~ rewriting\n",
|
||||||
"steps = NewtonRoot(f, df, x0=1.0).trace()\n",
|
"g = lambda x: (2/x) # not always convergent, but demonstrates\n",
|
||||||
"print(\"Newton Method Trace (x^2 - 2):\")\n",
|
"try:\n",
|
||||||
"print_trace(steps)\n",
|
" fp = FixedPoint(g, 1.0, tol=1e-8)\n",
|
||||||
|
" root_fp = fp.solve()\n",
|
||||||
|
" print('Fixed-point root:', root_fp)\n",
|
||||||
|
"except Exception as e:\n",
|
||||||
|
" print('Fixed-point failed:', e)\n",
|
||||||
"\n",
|
"\n",
|
||||||
"# Secant\n",
|
|
||||||
"steps = Secant(f, 0, 2).trace()\n",
|
|
||||||
"print(\"\\nSecant Method Trace (x^2 - 2):\")\n",
|
|
||||||
"print_trace(steps)\n",
|
|
||||||
"\n",
|
|
||||||
"# Bisection\n",
|
|
||||||
"steps = Bisection(f, 0, 2).trace()\n",
|
|
||||||
"print(\"\\nBisection Method Trace (x^2 - 2):\")\n",
|
|
||||||
"print_trace(steps)\n",
|
|
||||||
"\n",
|
|
||||||
"# Fixed-point: solve\n",
|
|
||||||
"g = lambda x: 0.5 * (x + 2 / x)\n",
|
"g = lambda x: 0.5 * (x + 2 / x)\n",
|
||||||
"steps = FixedPoint(g, 1.0).trace()\n",
|
"steps = FixedPoint(g, 1.0).trace()\n",
|
||||||
"print(\"\\nFixed-Point Iteration Trace (x^2 - 2):\")\n",
|
"print(\"\\nFixed-Point Iteration Trace (x^2 - 2):\")\n",
|
||||||
"print_trace(steps)"
|
"print_trace(steps)"
|
||||||
]
|
]
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "40e66a30",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## 4. Newton’s Method\n",
|
||||||
|
"From Taylor expansion:\n",
|
||||||
|
"$$\n",
|
||||||
|
"f(x) \\approx f(x_k) + f'(x_k)(x-x_k).\n",
|
||||||
|
"$$\n",
|
||||||
|
"Set $f(x)=0$ to solve for next iterate:\n",
|
||||||
|
"$$\n",
|
||||||
|
"x_{k+1} = x_k - \\frac{f(x_k)}{f'(x_k)}.\n",
|
||||||
|
"$$\n",
|
||||||
|
"\n",
|
||||||
|
"- **Quadratic convergence** for simple roots.\n",
|
||||||
|
"- For multiple roots: drops to linear.\n",
|
||||||
|
"- Requires derivative, sensitive to initial guess.\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 4,
|
||||||
|
"id": "7ebf9068",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Newton root: 1.4142135623730951\n",
|
||||||
|
"Newton Method Trace (x^2 - 2):\n",
|
||||||
|
" iter | x | f(x) | df(x) | x_new | error\n",
|
||||||
|
"------------------------------------------------------------------------------\n",
|
||||||
|
" 0 | 1 | -1 | 2 | 1.5 | 0.5\n",
|
||||||
|
" 1 | 1.5 | 0.25 | 3 | 1.41667 | 0.0833333\n",
|
||||||
|
" 2 | 1.41667 | 0.00694444 | 2.83333 | 1.41422 | 0.00245098\n",
|
||||||
|
" 3 | 1.41422 | 6.0073e-06 | 2.82843 | 1.41421 | 2.1239e-06\n",
|
||||||
|
" 4 | 1.41421 | 4.51061e-12 | 2.82843 | 1.41421 | 1.59472e-12\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"# Newton\n",
|
||||||
|
"newton = NewtonRoot(f, df, 1.0, tol=1e-12)\n",
|
||||||
|
"root_n = newton.solve()\n",
|
||||||
|
"print('Newton root:', root_n)\n",
|
||||||
|
"\n",
|
||||||
|
"steps = newton.trace()\n",
|
||||||
|
"print(\"Newton Method Trace (x^2 - 2):\")\n",
|
||||||
|
"print_trace(steps)"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "84888305",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## 5. Secant Method\n",
|
||||||
|
"- Avoids derivative by approximating slope with finite difference:\n",
|
||||||
|
"$$\n",
|
||||||
|
"x_{k+1} = x_k - f(x_k) \\frac{x_k - x_{k-1}}{f(x_k)-f(x_{k-1})}.\n",
|
||||||
|
"$$\n",
|
||||||
|
"\n",
|
||||||
|
"- Convergence order: $\\approx 1.618$ (superlinear).\n",
|
||||||
|
"- More efficient than Newton if derivative expensive.\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "code",
|
||||||
|
"execution_count": 5,
|
||||||
|
"id": "f2318bf3",
|
||||||
|
"metadata": {},
|
||||||
|
"outputs": [
|
||||||
|
{
|
||||||
|
"name": "stdout",
|
||||||
|
"output_type": "stream",
|
||||||
|
"text": [
|
||||||
|
"Secant root: 1.414213562373095\n",
|
||||||
|
"\n",
|
||||||
|
"Secant Method Trace (x^2 - 2):\n",
|
||||||
|
" iter | x0 | x1 | x2 | f(x0) | f(x1) | error\n",
|
||||||
|
"-------------------------------------------------------------------------------------------\n",
|
||||||
|
" 0 | 0 | 2 | 1 | -2 | 2 | 1\n",
|
||||||
|
" 1 | 2 | 1 | 1.33333 | 2 | -1 | 0.333333\n",
|
||||||
|
" 2 | 1 | 1.33333 | 1.42857 | -1 | -0.222222 | 0.0952381\n",
|
||||||
|
" 3 | 1.33333 | 1.42857 | 1.41379 | -0.222222 | 0.0408163 | 0.0147783\n",
|
||||||
|
" 4 | 1.42857 | 1.41379 | 1.41421 | 0.0408163 | -0.00118906 | 0.000418335\n",
|
||||||
|
" 5 | 1.41379 | 1.41421 | 1.41421 | -0.00118906 | -6.00729e-06 | 2.12421e-06\n",
|
||||||
|
" 6 | 1.41421 | 1.41421 | 1.41421 | -6.00729e-06 | 8.93146e-10 | 3.15775e-10\n",
|
||||||
|
" 7 | 1.41421 | 1.41421 | 1.41421 | 8.93146e-10 | -8.88178e-16 | 2.22045e-16\n"
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"source": [
|
||||||
|
"# Secant\n",
|
||||||
|
"sec = Secant(f, 0, 2, tol=1e-12)\n",
|
||||||
|
"root_s = sec.solve()\n",
|
||||||
|
"print('Secant root:', root_s)\n",
|
||||||
|
"\n",
|
||||||
|
"steps = sec.trace()\n",
|
||||||
|
"print(\"\\nSecant Method Trace (x^2 - 2):\")\n",
|
||||||
|
"print_trace(steps)\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"cell_type": "markdown",
|
||||||
|
"id": "7ed8a0b5",
|
||||||
|
"metadata": {},
|
||||||
|
"source": [
|
||||||
|
"## 6. Stopping Criteria\n",
|
||||||
|
"We stop iteration when:\n",
|
||||||
|
"- $|f(x_k)| \\le \\varepsilon$ (residual small), or\n",
|
||||||
|
"- $|x_{k+1}-x_k| \\le \\varepsilon(1+|x_{k+1}|)$ (relative step small).\n"
|
||||||
|
]
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"cell_type": "markdown",
|
"cell_type": "markdown",
|
||||||
"id": "833b538c",
|
"id": "833b538c",
|
||||||
|
|||||||
Reference in New Issue
Block a user