1---2name: nonlinear-solvers3description: >4 Select and configure nonlinear solvers for root-finding f(x)=0, optimization5 min F(x), and least-squares problems — choose among Newton, Newton-Krylov,6 quasi-Newton (BFGS, L-BFGS), Broyden, Anderson acceleration, and7 Levenberg-Marquardt methods, configure line search or trust-region8 globalization, diagnose convergence rate (quadratic, linear, stagnated),9 and assess Jacobian quality and conditioning. Use when a Newton solver10 converges slowly or diverges, choosing between line search and trust region,11 debugging nonlinear iteration failures in FEM or phase-field codes, or12 selecting a solver for large-scale unconstrained optimization, even if13 the user only says "my Newton iterations aren't converging."14allowed-tools: Read, Bash, Write, Grep, Glob15metadata:16 source: https://skillmd.com/skills/heshamfs/nonlinear-solvers17 author: HeshamFS18 version: "1.2.2"19 security_tier: high20 security_reviewed: true21 tested_with:22 - claude-code23 last_evaluated: "2026-06-24"24 eval_cases: 525 last_reviewed: "2026-06-23"26 standards:27 - "Nocedal & Wright (2006), Numerical Optimization (BFGS/L-BFGS, trust region, dogleg, Steihaug-CG, Wolfe conditions)"28 - "Dennis & Schnabel (1996), Numerical Methods for Unconstrained Optimization and Nonlinear Equations (Newton, Broyden good/bad, line-search globalization)"29 - "Kelley (1995), Iterative Methods for Linear and Nonlinear Equations (inexact Newton, Newton-Krylov); Eisenstat & Walker (1996) forcing sequence"30 - "Walker & Ni (2011), Anderson acceleration for fixed-point iterations (Anderson 1965)"31 - "Levenberg (1944) / Marquardt (1963), Levenberg-Marquardt method for nonlinear least-squares"32---3334# Nonlinear Solvers3536## Goal3738Provide a universal workflow to select a nonlinear solver, configure globalization strategies, and diagnose convergence for root-finding, optimization, and least-squares problems.3940## Requirements4142- Python 3.10+43- NumPy (for Jacobian diagnostics)44- SciPy (optional, for advanced analysis)4546## Inputs to Gather4748| Input | Description | Example |49|-------|-------------|---------|50| Problem type | Root-finding, optimization, least-squares | `root-finding` |51| Problem size | Number of unknowns | `n = 10000` |52| Jacobian availability | Analytic, finite-diff, unavailable | `analytic` |53| Jacobian cost | Cheap or expensive to compute | `expensive` |54| Constraints | None, bounds, equality, inequality | `none` |55| Smoothness | Is objective/residual smooth? | `yes` |56| Residual history | Sequence of residual norms | `1,0.1,0.01,...` |5758## Decision Guidance5960### Solver Selection Flowchart6162```63Is Jacobian available and cheap?64├── YES → Problem size?65│ ├── Small (n < 1000) → Newton (full)66│ └── Large (n ≥ 1000) → Newton-Krylov67└── NO → Is objective smooth?68 ├── YES → Memory limited?69 │ ├── YES → L-BFGS or Broyden70 │ └── NO → BFGS71 └── NO → Anderson acceleration or Picard72```7374### Quick Reference7576| Problem Type | First Choice | Alternative | Globalization |77|--------------|--------------|-------------|---------------|78| Small root-finding | Newton | Broyden | Line search |79| Large root-finding | Newton-Krylov | Anderson | Trust region |80| Optimization | L-BFGS | BFGS | Wolfe line search |81| Least-squares | Levenberg-Marquardt | Gauss-Newton | Trust region |82| Bound constrained | L-BFGS-B | Trust-region reflective | Projected |8384## Script Outputs (JSON Fields)8586| Script | Key Outputs |87|--------|-------------|88| `scripts/solver_selector.py` | `recommended`, `alternatives`, `notes` |89| `scripts/convergence_analyzer.py` | `converged`, `convergence_type`, `estimated_rate`, `diagnosis` |90| `scripts/jacobian_diagnostics.py` | `condition_number`, `jacobian_quality`, `rank_deficient` |91| `scripts/globalization_advisor.py` | `strategy`, `line_search_type`, `trust_region_type`, `parameters` |92| `scripts/residual_monitor.py` | `patterns_detected`, `alerts`, `recommendations` |93| `scripts/step_quality.py` | `ratio`, `step_quality`, `accept_step`, `trust_radius_action` |9495## Workflow96971. **Characterize problem** - Identify type, size, Jacobian availability982. **Select solver** - Run `scripts/solver_selector.py`993. **Choose globalization** - Run `scripts/globalization_advisor.py`1004. **Analyze Jacobian** - If available, run `scripts/jacobian_diagnostics.py`1015. **Monitor residuals** - During solve, use `scripts/residual_monitor.py`1026. **Analyze convergence** - Run `scripts/convergence_analyzer.py`1037. **Evaluate steps** - For trust region, use `scripts/step_quality.py`104105## Conversational Workflow Example106107**User**: My Newton solver for a phase-field simulation is converging very slowly. After 50 iterations, the residual only dropped from 1 to 0.1.108109**Agent workflow**:1101. Analyze convergence:111 ```bash112 python3 scripts/convergence_analyzer.py --residuals 1,0.8,0.6,0.5,0.4,0.3,0.2,0.15,0.12,0.1 --json113 ```1142. Check globalization strategy:115 ```bash116 python3 scripts/globalization_advisor.py --problem-type root-finding --jacobian-quality ill-conditioned --previous-failures 0 --json117 ```1183. Recommend: Switch to trust region with Levenberg-Marquardt regularization, or use Newton-Krylov with better preconditioning.119120## Pre-Solve Checklist121122- [ ] Confirm problem type (root-finding, optimization, least-squares)123- [ ] Assess Jacobian availability and cost124- [ ] Check initial guess quality125- [ ] Set appropriate tolerances126- [ ] Choose globalization strategy127- [ ] Prepare to monitor convergence128129## CLI Examples130131```bash132# Select solver for large unconstrained optimization133python3 scripts/solver_selector.py --size 50000 --smooth --memory-limited --json134135# Select solver for a small nonlinear least-squares (data-fitting) problem136python3 scripts/solver_selector.py --problem-type least-squares --size 6 --jacobian-available --smooth --json137138# Analyze convergence from residual history139python3 scripts/convergence_analyzer.py --residuals 1,0.1,0.01,0.001,0.0001 --tolerance 1e-6 --json140141# Diagnose Jacobian quality142python3 scripts/jacobian_diagnostics.py --matrix jacobian.txt --json143144# Get globalization recommendation145python3 scripts/globalization_advisor.py --problem-type optimization --jacobian-quality good --json146147# Globalization for a distant initial guess (favors trust region)148python3 scripts/globalization_advisor.py --problem-type root-finding --jacobian-quality good --far-from-solution --json149150# Monitor residual patterns151python3 scripts/residual_monitor.py --residuals 1,0.8,0.9,0.7,0.75,0.6 --target-tolerance 1e-8 --json152153# Evaluate step quality for trust region154python3 scripts/step_quality.py --predicted-reduction 0.5 --actual-reduction 0.4 --step-norm 0.8 --gradient-norm 1.0 --trust-radius 1.0 --json155```156157## Error Handling158159| Error | Cause | Resolution |160|-------|-------|------------|161| `problem_size must be positive` | Invalid size | Check problem dimension |162| `problem_size (...) exceeds maximum (...)` | Size above 10 billion cap | Re-check the unit/value |163| `constraint_type must be one of...` | Unknown constraint | Use: none, bound, equality, inequality |164| `problem_type must be one of...` | Unknown problem type | Use: root-finding, optimization, least-squares |165| `residuals must be non-negative` | Invalid residual data | Check residual computation |166| `residuals must be finite` | NaN/Inf in residual data | Sanitize residual history |167| `residual list length (...) exceeds limit (...)` | More than 100,000 entries | Downsample the history |168| `Matrix file not found` | Invalid path | Verify Jacobian file exists |169| `Matrix file exceeds size limit ...` | Matrix file too large | Use a smaller / sparser matrix |170171## Interpretation Guidance172173### Convergence Type174175| Type | Meaning | Action |176|------|---------|--------|177| quadratic | Optimal Newton (order p ≈ 2) | Continue, near solution |178| superlinear | Ratios shrinking toward 0 (1 < p < 2); quasi-Newton working | Monitor for stagnation |179| linear | Constant contraction ratio (p ≈ 1); a small constant ratio is fast-linear, not superlinear | May improve with preconditioner |180| sublinear | Too slow (ratio → 1) | Change method or formulation |181| stagnated | No progress | Check Jacobian, preconditioner |182| diverged | Increasing residual | Add globalization, check Jacobian |183184### Jacobian Quality185186| Quality | Condition Number | Action |187|---------|------------------|--------|188| good | < 10⁶ | Standard Newton works |189| moderately-conditioned | 10⁶ - 10¹⁰ | Consider scaling |190| ill-conditioned | > 10¹⁰ | Use regularization |191| near-singular | ∞ | Reformulate or use LM |192193### Step Quality (Trust Region)194195| Ratio ρ | Quality | Trust Radius |196|---------|---------|--------------|197| ρ < 0 | very_poor | Shrink aggressively |198| ρ < 0.25 | marginal | Shrink |199| 0.25 ≤ ρ < 0.75 | good | Maintain |200| ρ ≥ 0.75 | excellent | Expand if at boundary |201202## Verification checklist203204Do not trust a "solved" claim until these concrete artifacts are recorded:205206- [ ] Logged the full residual norm history and ran `convergence_analyzer.py --residuals <history>`; recorded `convergence_type` and `estimated_rate`, and confirmed `converged: true` against the actual solver tolerance (not the default `1e-10`).207- [ ] Confirmed the residual sequence is monotone-decreasing or fed it to `residual_monitor.py`; recorded `patterns_detected` and verified it does NOT include `diverging`, `oscillating`, `plateau`, or `slow_convergence` while still above tolerance.208- [ ] If a Jacobian is available, ran `jacobian_diagnostics.py --matrix J.txt` and recorded `condition_number` and `jacobian_quality`; for an analytic Jacobian, passed `--finite-diff-matrix` and confirmed `finite_diff_error` is below ~1e-2 (no "Large discrepancy" note).209- [ ] Checked `rank_deficient` from `jacobian_diagnostics.py` is `false` (or documented why a rank-deficient/near-singular Jacobian is expected and that Levenberg-Marquardt regularization is in use).210- [ ] For a trust-region solve, evaluated accepted steps with `step_quality.py` and recorded the reduction `ratio`; confirmed accepted steps have `ratio >= 0.25` (not `very_poor`/`poor`) and that the `trust_radius_action` matches the recorded ρ.211- [ ] Recorded the solver and globalization actually used and confirmed they match `solver_selector.py` and `globalization_advisor.py` recommendations for the stated problem type, size, and Jacobian quality (e.g., large/expensive-Jacobian → Newton-Krylov; least-squares → Levenberg-Marquardt trust region).212- [ ] Re-confirmed convergence after any change to tolerance, initial guess, or preconditioner — the convergence type can flip (e.g., quadratic → linear/stagnated) and must be re-classified, not assumed.213214## Common pitfalls & rationalizations215216| Tempting shortcut | Why it's wrong / what to do |217|-------------------|-----------------------------|218| "The residual ratio is a small constant (~0.1), so it's converging superlinearly." | A *constant* contraction ratio is linear, not superlinear — `convergence_analyzer.py` reports this as `linear` (annotated "fast linear"). Superlinear requires the ratio to tend to zero (order p > 1.2). Don't claim Newton-quality convergence from a flat ratio. |219| "It stopped without erroring, so the solver converged." | Run completion is not convergence. Check `converged` from `convergence_analyzer.py`/`residual_monitor.py` against the real tolerance; a `stagnated` or `plateau` result also "stops" but has not solved `f(x)=0`. |220| "Two iterations look like they're shrinking, so the rate is fine." | Order estimation needs at least 3 strictly decreasing positive residuals; with fewer, `convergence_analyzer.py` returns `unknown`/falls back to rate-only. Gather more iterations before quoting a convergence type. |221| "I coded the analytic Jacobian, so it must be right." | A wrong Jacobian still produces *some* step. Run `jacobian_diagnostics.py --finite-diff-matrix` and confirm `finite_diff_error` is small; a "Large discrepancy with finite-diff" note means the analytic Jacobian is buggy, which silently degrades Newton to linear convergence. |222| "Newton diverged, so I'll just shrink the global tolerance and call it close enough." | Divergence (`convergence_type: diverged`, or `diverging` pattern) signals a bad step direction or far-from-solution start — add globalization. Run `globalization_advisor.py` (use `--far-from-solution` / report failures) and switch to a trust region or damped step instead of loosening the target. |223| "Trust-region step decreased the objective, so accept and expand the radius." | Acceptance and radius growth depend on the reduction ratio ρ, not just sign. `step_quality.py` only flags `expand` when ρ ≥ 0.75 *and* the step hit the boundary; a small positive ρ (`marginal`) means accept-but-shrink. Use the recorded `trust_radius_action`. |224| "The Jacobian is large and expensive, but full Newton is the gold standard, so I'll form it anyway." | For n ≥ 1000 or expensive Jacobians, `solver_selector.py` routes to matrix-free Newton-Krylov (JFNK) precisely because forming/factoring J is infeasible; use Jacobian-vector products plus a preconditioner instead. |225226## Security227228### Input Validation229- `--size` (problem size) is validated as a positive integer, bounded at 10 billion230- `--residuals` are validated as finite non-negative numbers, capped at 100,000 entries231- `--tolerance` and `--target-tolerance` are validated as finite positive numbers232- `--problem-type` and `--constraint-type` are validated against fixed allowlists233- `--jacobian-quality` is validated against a fixed allowlist (`good`, `ill-conditioned`, etc.)234- Step quality parameters (`predicted-reduction`, `actual-reduction`, `step-norm`, `gradient-norm`, `trust-radius`) are validated as finite numbers235236### File Access237- `jacobian_diagnostics.py` reads a single matrix file specified by `--matrix`; no directory traversal beyond the given path238- Matrix files are size-limited and loaded with `allow_pickle=False` to prevent code execution239- All other scripts read no external files; inputs are provided via CLI arguments240- Scripts write only to stdout (JSON output)241242### Tool Restrictions243- **Read**: Used to inspect script source, references, and user configuration files244- **Bash**: Used to execute the six Python analysis scripts (`solver_selector.py`, `convergence_analyzer.py`, `jacobian_diagnostics.py`, `globalization_advisor.py`, `residual_monitor.py`, `step_quality.py`) with explicit argument lists245- **Write**: Used to save analysis results or solver recommendations; writes are scoped to the user's working directory246- **Grep/Glob**: Used to locate relevant files and search references247248### Safety Measures249- No `eval()`, `exec()`, or dynamic code generation250- All subprocess calls use explicit argument lists (no `shell=True`)251- Matrix dimension limits prevent memory exhaustion when loading Jacobian files252- Residual history analysis operates on bounded-length numeric arrays only253254## Limitations255256- **No global convergence guarantee**: All methods may fail for pathological problems257- **Jacobian accuracy**: Finite-difference Jacobian may be inaccurate near discontinuities258- **Large dense problems**: May require specialized solvers not covered here259- **Constrained optimization**: Complex constraints need SQP or interior point methods260261## References262263- `references/solver_decision_tree.md` - Problem-based solver selection264- `references/method_catalog.md` - Method details and parameters265- `references/convergence_diagnostics.md` - Diagnosing convergence issues266- `references/globalization_strategies.md` - Line search and trust region267268## Version History269270- **v1.2.2** (2026-06-24): Added a "Verification checklist" (evidence tied to each script's JSON outputs — convergence type/rate, residual patterns, Jacobian condition/finite-diff error, rank, trust-region step ratio, and solver/globalization agreement) and a "Common pitfalls & rationalizations" table covering constant-ratio-vs-superlinear, run-completion-vs-convergence, too-few-iterations, unverified analytic Jacobians, divergence handling, trust-region acceptance, and large/expensive-Jacobian routing271- **v1.2.0** (2026-06-23): Added `--problem-type` to `solver_selector.py` with a nonlinear least-squares path (Levenberg-Marquardt / Gauss-Newton); reordered solver selection so problem size dominates high-accuracy and routes large/expensive-Jacobian problems to Newton-Krylov; added `--far-from-solution` to `globalization_advisor.py` and surfaced Levenberg-Marquardt as the trust-region type for least-squares; corrected convergence classification so constant-ratio sequences are linear (not superlinear); RFC-8259-safe JSON (no `-Infinity`); input-validation hardening272- **v1.1.0** (2026-03-26): Optimized agent-discovery description, evaluation suite, security review docs, standardized metadata block, CHANGELOG273- **v1.0.0**: Initial release with 6 analysis scripts