Nonlinear Solvers
Goal
Provide a universal workflow to select a nonlinear solver, configure globalization strategies, and diagnose convergence for root-finding, optimization, and least-squares problems.
Requirements
- Python 3.8+
- NumPy (for Jacobian diagnostics)
- SciPy (optional, for advanced analysis)
Inputs to Gather
| Input |
Description |
Example |
| Problem type |
Root-finding, optimization, least-squares |
root-finding |
| Problem size |
Number of unknowns |
n = 10000 |
| Jacobian availability |
Analytic, finite-diff, unavailable |
analytic |
| Jacobian cost |
Cheap or expensive to compute |
expensive |
| Constraints |
None, bounds, equality, inequality |
none |
| Smoothness |
Is objective/residual smooth? |
yes |
| Residual history |
Sequence of residual norms |
1,0.1,0.01,... |
Decision Guidance
Solver Selection Flowchart
Is Jacobian available and cheap?
├── YES → Problem size?
│ ├── Small (n < 1000) → Newton (full)
│ └── Large (n ≥ 1000) → Newton-Krylov
└── NO → Is objective smooth?
├── YES → Memory limited?
│ ├── YES → L-BFGS or Broyden
│ └── NO → BFGS
└── NO → Anderson acceleration or Picard
Quick Reference
| Problem Type |
First Choice |
Alternative |
Globalization |
| Small root-finding |
Newton |
Broyden |
Line search |
| Large root-finding |
Newton-Krylov |
Anderson |
Trust region |
| Optimization |
L-BFGS |
BFGS |
Wolfe line search |
| Least-squares |
Levenberg-Marquardt |
Gauss-Newton |
Trust region |
| Bound constrained |
L-BFGS-B |
Trust-region reflective |
Projected |
Script Outputs (JSON Fields)
| Script |
Key Outputs |
scripts/solver_selector.py |
recommended, alternatives, notes |
scripts/convergence_analyzer.py |
converged, convergence_type, estimated_rate, diagnosis |
scripts/jacobian_diagnostics.py |
condition_number, jacobian_quality, rank_deficient |
scripts/globalization_advisor.py |
strategy, line_search_type, trust_region_type, parameters |
scripts/residual_monitor.py |
patterns_detected, alerts, recommendations |
scripts/step_quality.py |
ratio, step_quality, accept_step, trust_radius_action |
Workflow
- Characterize problem - Identify type, size, Jacobian availability
- Select solver - Run
scripts/solver_selector.py
- Choose globalization - Run
scripts/globalization_advisor.py
- Analyze Jacobian - If available, run
scripts/jacobian_diagnostics.py
- Monitor residuals - During solve, use
scripts/residual_monitor.py
- Analyze convergence - Run
scripts/convergence_analyzer.py
- Evaluate steps - For trust region, use
scripts/step_quality.py
Conversational Workflow Example
User: My Newton solver for a phase-field simulation is converging very slowly. After 50 iterations, the residual only dropped from 1 to 0.1.
Agent workflow:
- Analyze convergence:
python3 scripts/convergence_analyzer.py --residuals 1,0.8,0.6,0.5,0.4,0.3,0.2,0.15,0.12,0.1 --json
- Check globalization strategy:
python3 scripts/globalization_advisor.py --problem-type root-finding --jacobian-quality ill-conditioned --previous-failures 0 --json
- Recommend: Switch to trust region with Levenberg-Marquardt regularization, or use Newton-Krylov with better preconditioning.
Pre-Solve Checklist
CLI Examples
# Select solver for large unconstrained optimization
python3 scripts/solver_selector.py --size 50000 --smooth --memory-limited --json
# Analyze convergence from residual history
python3 scripts/convergence_analyzer.py --residuals 1,0.1,0.01,0.001,0.0001 --tolerance 1e-6 --json
# Diagnose Jacobian quality
python3 scripts/jacobian_diagnostics.py --matrix jacobian.txt --json
# Get globalization recommendation
python3 scripts/globalization_advisor.py --problem-type optimization --jacobian-quality good --json
# Monitor residual patterns
python3 scripts/residual_monitor.py --residuals 1,0.8,0.9,0.7,0.75,0.6 --target-tolerance 1e-8 --json
# Evaluate step quality for trust region
python3 scripts/step_quality.py --predicted-reduction 0.5 --actual-reduction 0.4 --step-norm 0.8 --gradient-norm 1.0 --trust-radius 1.0 --json
Error Handling
| Error |
Cause |
Resolution |
problem_size must be positive |
Invalid size |
Check problem dimension |
constraint_type must be one of... |
Unknown constraint |
Use: none, bound, equality, inequality |
residuals must be non-negative |
Invalid residual data |
Check residual computation |
Matrix file not found |
Invalid path |
Verify Jacobian file exists |
Interpretation Guidance
Convergence Type
| Type |
Meaning |
Action |
| quadratic |
Optimal Newton |
Continue, near solution |
| superlinear |
Quasi-Newton working |
Monitor for stagnation |
| linear |
Acceptable |
May improve with preconditioner |
| sublinear |
Too slow |
Change method or formulation |
| stagnated |
No progress |
Check Jacobian, preconditioner |
| diverged |
Increasing residual |
Add globalization, check Jacobian |
Jacobian Quality
| Quality |
Condition Number |
Action |
| good |
< 10⁶ |
Standard Newton works |
| moderately-conditioned |
10⁶ - 10¹⁰ |
Consider scaling |
| ill-conditioned |
> 10¹⁰ |
Use regularization |
| near-singular |
∞ |
Reformulate or use LM |
Step Quality (Trust Region)
| Ratio ρ |
Quality |
Trust Radius |
| ρ < 0 |
very_poor |
Shrink aggressively |
| ρ < 0.25 |
marginal |
Shrink |
| 0.25 ≤ ρ < 0.75 |
good |
Maintain |
| ρ ≥ 0.75 |
excellent |
Expand if at boundary |
Limitations
- No global convergence guarantee: All methods may fail for pathological problems
- Jacobian accuracy: Finite-difference Jacobian may be inaccurate near discontinuities
- Large dense problems: May require specialized solvers not covered here
- Constrained optimization: Complex constraints need SQP or interior point methods
References
references/solver_decision_tree.md - Problem-based solver selectionreferences/method_catalog.md - Method details and parametersreferences/convergence_diagnostics.md - Diagnosing convergence issuesreferences/globalization_strategies.md - Line search and trust region
Version History
- v1.0.0 : Initial release with 6 analysis scripts
1---2name: nonlinear-solvers3description: Select and configure nonlinear solvers for f(x)=0 or min F(x). Use for Newton methods, quasi-Newton (BFGS, L-BFGS), Broyden, Anderson acceleration, diagnosing convergence issues, choosing line search vs trust region, and analyzing Jacobian quality.4---56# Nonlinear Solvers78## Goal910Provide a universal workflow to select a nonlinear solver, configure globalization strategies, and diagnose convergence for root-finding, optimization, and least-squares problems.1112## Requirements1314- Python 3.8+15- NumPy (for Jacobian diagnostics)16- SciPy (optional, for advanced analysis)1718## Inputs to Gather1920| Input | Description | Example |21|-------|-------------|---------|22| Problem type | Root-finding, optimization, least-squares | `root-finding` |23| Problem size | Number of unknowns | `n = 10000` |24| Jacobian availability | Analytic, finite-diff, unavailable | `analytic` |25| Jacobian cost | Cheap or expensive to compute | `expensive` |26| Constraints | None, bounds, equality, inequality | `none` |27| Smoothness | Is objective/residual smooth? | `yes` |28| Residual history | Sequence of residual norms | `1,0.1,0.01,...` |2930## Decision Guidance3132### Solver Selection Flowchart3334```35Is Jacobian available and cheap?36├── YES → Problem size?37│ ├── Small (n < 1000) → Newton (full)38│ └── Large (n ≥ 1000) → Newton-Krylov39└── NO → Is objective smooth?40 ├── YES → Memory limited?41 │ ├── YES → L-BFGS or Broyden42 │ └── NO → BFGS43 └── NO → Anderson acceleration or Picard44```4546### Quick Reference4748| Problem Type | First Choice | Alternative | Globalization |49|--------------|--------------|-------------|---------------|50| Small root-finding | Newton | Broyden | Line search |51| Large root-finding | Newton-Krylov | Anderson | Trust region |52| Optimization | L-BFGS | BFGS | Wolfe line search |53| Least-squares | Levenberg-Marquardt | Gauss-Newton | Trust region |54| Bound constrained | L-BFGS-B | Trust-region reflective | Projected |5556## Script Outputs (JSON Fields)5758| Script | Key Outputs |59|--------|-------------|60| `scripts/solver_selector.py` | `recommended`, `alternatives`, `notes` |61| `scripts/convergence_analyzer.py` | `converged`, `convergence_type`, `estimated_rate`, `diagnosis` |62| `scripts/jacobian_diagnostics.py` | `condition_number`, `jacobian_quality`, `rank_deficient` |63| `scripts/globalization_advisor.py` | `strategy`, `line_search_type`, `trust_region_type`, `parameters` |64| `scripts/residual_monitor.py` | `patterns_detected`, `alerts`, `recommendations` |65| `scripts/step_quality.py` | `ratio`, `step_quality`, `accept_step`, `trust_radius_action` |6667## Workflow68691. **Characterize problem** - Identify type, size, Jacobian availability702. **Select solver** - Run `scripts/solver_selector.py`713. **Choose globalization** - Run `scripts/globalization_advisor.py`724. **Analyze Jacobian** - If available, run `scripts/jacobian_diagnostics.py`735. **Monitor residuals** - During solve, use `scripts/residual_monitor.py`746. **Analyze convergence** - Run `scripts/convergence_analyzer.py`757. **Evaluate steps** - For trust region, use `scripts/step_quality.py`7677## Conversational Workflow Example7879**User**: My Newton solver for a phase-field simulation is converging very slowly. After 50 iterations, the residual only dropped from 1 to 0.1.8081**Agent workflow**:821. Analyze convergence:83 ```bash84 python3 scripts/convergence_analyzer.py --residuals 1,0.8,0.6,0.5,0.4,0.3,0.2,0.15,0.12,0.1 --json85 ```862. Check globalization strategy:87 ```bash88 python3 scripts/globalization_advisor.py --problem-type root-finding --jacobian-quality ill-conditioned --previous-failures 0 --json89 ```903. Recommend: Switch to trust region with Levenberg-Marquardt regularization, or use Newton-Krylov with better preconditioning.9192## Pre-Solve Checklist9394- [ ] Confirm problem type (root-finding, optimization, least-squares)95- [ ] Assess Jacobian availability and cost96- [ ] Check initial guess quality97- [ ] Set appropriate tolerances98- [ ] Choose globalization strategy99- [ ] Prepare to monitor convergence100101## CLI Examples102103```bash104# Select solver for large unconstrained optimization105python3 scripts/solver_selector.py --size 50000 --smooth --memory-limited --json106107# Analyze convergence from residual history108python3 scripts/convergence_analyzer.py --residuals 1,0.1,0.01,0.001,0.0001 --tolerance 1e-6 --json109110# Diagnose Jacobian quality111python3 scripts/jacobian_diagnostics.py --matrix jacobian.txt --json112113# Get globalization recommendation114python3 scripts/globalization_advisor.py --problem-type optimization --jacobian-quality good --json115116# Monitor residual patterns117python3 scripts/residual_monitor.py --residuals 1,0.8,0.9,0.7,0.75,0.6 --target-tolerance 1e-8 --json118119# Evaluate step quality for trust region120python3 scripts/step_quality.py --predicted-reduction 0.5 --actual-reduction 0.4 --step-norm 0.8 --gradient-norm 1.0 --trust-radius 1.0 --json121```122123## Error Handling124125| Error | Cause | Resolution |126|-------|-------|------------|127| `problem_size must be positive` | Invalid size | Check problem dimension |128| `constraint_type must be one of...` | Unknown constraint | Use: none, bound, equality, inequality |129| `residuals must be non-negative` | Invalid residual data | Check residual computation |130| `Matrix file not found` | Invalid path | Verify Jacobian file exists |131132## Interpretation Guidance133134### Convergence Type135136| Type | Meaning | Action |137|------|---------|--------|138| quadratic | Optimal Newton | Continue, near solution |139| superlinear | Quasi-Newton working | Monitor for stagnation |140| linear | Acceptable | May improve with preconditioner |141| sublinear | Too slow | Change method or formulation |142| stagnated | No progress | Check Jacobian, preconditioner |143| diverged | Increasing residual | Add globalization, check Jacobian |144145### Jacobian Quality146147| Quality | Condition Number | Action |148|---------|------------------|--------|149| good | < 10⁶ | Standard Newton works |150| moderately-conditioned | 10⁶ - 10¹⁰ | Consider scaling |151| ill-conditioned | > 10¹⁰ | Use regularization |152| near-singular | ∞ | Reformulate or use LM |153154### Step Quality (Trust Region)155156| Ratio ρ | Quality | Trust Radius |157|---------|---------|--------------|158| ρ < 0 | very_poor | Shrink aggressively |159| ρ < 0.25 | marginal | Shrink |160| 0.25 ≤ ρ < 0.75 | good | Maintain |161| ρ ≥ 0.75 | excellent | Expand if at boundary |162163## Limitations164165- **No global convergence guarantee**: All methods may fail for pathological problems166- **Jacobian accuracy**: Finite-difference Jacobian may be inaccurate near discontinuities167- **Large dense problems**: May require specialized solvers not covered here168- **Constrained optimization**: Complex constraints need SQP or interior point methods169170## References171172- `references/solver_decision_tree.md` - Problem-based solver selection173- `references/method_catalog.md` - Method details and parameters174- `references/convergence_diagnostics.md` - Diagnosing convergence issues175- `references/globalization_strategies.md` - Line search and trust region176177## Version History178179- **v1.0.0** : Initial release with 6 analysis scripts
luokai0