Oasis Python Architecture

OASIS Python Architecture

• No duplicated logic (non-negotiable): Never copy-paste or parallel-implement the same behavior in two places. If something exists twice (even slightly divergent), stop and consolidate: one canonical implementation, callers import or delegate to it. This is mandatory DRY; KISS applies to the solution, not to skipping extraction. SOLID implies a single place per responsibility—duplication always violates that.
• Centralize by default: Shared constants, validation rules, formatters, parsers, JSON/report field naming, URL or path rules, and error messages belong in one module each (oasis/helpers/, oasis/schemas/, or another existing shared module)—not redefined ad hoc in multiple files.
• Keep responsibilities split by module: orchestration in oasis/oasis.py, analysis in oasis/analyze.py, structured schemas in oasis/schemas/, structured LLM output normalization / JSON repair in oasis/structured_output/ (deep.py, json_repair.py, json_repair_scan.py), canonical JSON + Jinja report rendering in oasis/report.py, export writers (Markdown/HTML/PDF/SARIF) in oasis/export/, dashboard indexing in oasis/web.py, model-backend lifecycle in oasis/backends/ (see below); oasis/ollama_manager.py is a backward-compat shim only.
• LangGraph (oasis/agent/): The compiled DAG (graph.py), state (state.py), stable node/route identifiers (graph_labels.py), and invoke_oasis_langgraph (invoke.py) own orchestration structure. Node bodies dispatch to SecurityAnalyzer methods in analyze.py via tools.py / nodes.py — keep business logic in analyze.py and keep the graph layer a thin wiring + state pass-through. The assistant validation subgraph (assistant_graph.py, assistant_labels.py, assistant_nodes.py, assistant_state.py, assistant_invoke.py) routes on vulnerability family (flow → entry_points → trace → taint → mitigations → verdict; access → entry_points → authz → verdict; config → config_audit → verdict) and follows the same lazy-import discipline. Use explicit imports from submodules (e.g. oasis.agent.invoke) so importing oasis.agent does not eagerly load LangGraph unless needed.
• Analysis type: AnalysisType.GRAPH is the only scan/deep orchestration mode; chunk caches use LangGraph paths only (oasis/cache.py).
• Model backends (oasis/backends/): base.ModelBackend is the provider-agnostic contract — chat / chat_stream / generate normalize responses to the Ollama shape ({"message": {"content": ...}}), and model listing, interactive selection, thinking overrides, and chunk-size detection live in the base. ollama_backend.OllamaManager keeps the Ollama-specific machinery (auto-pull, ps() runtime context, show() metadata, think=). openai_compat.OpenAICompatManager + OpenAICompatClient serve OpenAI-compatible servers (vLLM, LM Studio, llama.cpp, LiteLLM, …) via httpx while exposing ollama-shaped client methods (embeddings(model=, prompt=), list()), translating options.num_predict → max_tokens, format=<json_schema> → response_format with an automatic schema-in-prompt retry on HTTP 4xx (OASIS_OPENAI_STRUCTURED_OUTPUT), and reading context windows from OASIS_OPENAI_CTX_TOKENS. The factory create_model_manager (backends/__init__.py) resolves --provider → OASIS_LLM_PROVIDER → auto-detect, and multiprocessing worker namespaces carry provider/api_base/api_key. Never branch on the provider at call sites (analyze.py, embedding.py, web.py) — extend the backend contract instead; oasis/ollama_manager.py is a compat shim only. Tests: tests/test_backends_openai_compat.py.
• LangGraph-related helpers: oasis/helpers/langgraph_cli/ bundles CLI banners/emits, debug separators, LLM debug logging, and LangGraph vuln-type count helpers. oasis/helpers/progress/ includes graph pipeline rows (graph_pipeline_phases, graph_progress_extras, …), tqdm/coercion, executive-summary phase extras, and scan progress JSON/markdown helpers. oasis/helpers/context/expand.py holds expand_line_window / expand_suspicious_chunk_records (CONTEXT_EXPAND_* in oasis/config.py). oasis/helpers/poc/ covers PoC digest JSON, hints markdown, and PoC stage logging options. oasis/helpers/ollama_timing/ holds Ollama payload size / timeout helpers for logging.
• Embedding model normalization: Parse and normalize CLI embedding model values through oasis/helpers/embedding/ (normalize_embed_models, resolve_embed_models, primary_embed_model, resolve_valid_embedding_input_files) and keep one canonical source for model-list parsing and primary-model fallback. Invalid input raises EmbedModelValueError (subclass of ValueError); oasis/oasis.py maps that to argparse.ArgumentTypeError only in argparse type= callables.
• Helpers (oasis/helpers/): All reusable utilities that behave like helpers must live under oasis/helpers/, not inlined in feature modules (analyze.py, report.py, web.py, etc.). If you add or refactor something that is formatting, parsing, small pure transforms, progress/status builders, shared guards, or other cross-cutting non–entry-point utilities, move it into oasis/helpers/ as part of the change.
• Helper categories (thematic subpackages under oasis/helpers/): dashboard/ — audit_metrics, report_preview_html, dashboard_links, severity_filter, main __init__ (formats, Socket.IO/CORS, phase cells) and exec_summary_tiers (tiers; safe for early config import). phases/ — scan (phase rows, adaptive/standard phase lists). progress/ — tqdm, coercion, LangGraph pipeline extras, scan progress markdown (EXEC_SUMMARY_PROGRESS_EVENT_VERSION, SCAN_PROGRESS_EXTENDED_KEYS at top of oasis/helpers/progress/__init__.py). embedding/, langgraph_cli/, poc/, ollama_timing/, prompt_compose/, cli_update/. context/ — expand, path_containment (safe paths under security_reports). vuln/ — taxonomy, validation_patterns. executive/ — assistant scope, dashboard preview, modal chart metadata. report_project.py — project slug / output dir naming; analysis_root_path.py — canonical resolution of JSON analysis_root (relative under security_reports/ vs legacy absolute). executive_summary.py — executive canonical JSON + HTML view models. misc.py at package root (snippets + structured-output degeneracy); naming.py (shared naming rules) and report_jinja_filters.py (Jinja filters for report templates) sit at package root as well; lazy_export_validation.py backs the lazy-export contract (test_helpers_lazy_exports / test_helpers_lazy_subpackage_exports guard __all__ vs _LAZY_IMPORTS). Dashboard assistant under oasis/helpers/assistant/ (scan, authz, verdict, prompt, web, think). The web/ subpackage groups HTTP-orchestration helpers used by oasis/web.py: web_prepare.py (chat context preparation), sink_resolution.py (resolve (sink_file, sink_line) from finding indices + finding_scope_report_path; coerce_positive_int_line for numeric coercion), result_presentation.py (post-verdict EP / citation filter for flow + access families anchored on scope.sink_file; never mutates verdict), http_contract.py, rag.py, persistence.py, api_validate.py. Do not duplicate sink resolution or EP filtering logic inside web.py — call these helpers. schemas/audit_report.py — structured audit document beside Markdown. oasis/helpers/__init__.py uses lazy exports (test_helpers_lazy_exports guards __all__ vs _LAZY_IMPORTS).
• Report output layout: New scans default to security_reports/<project_slug>/…; CLI --project-name / -pn overrides slug derivation. Keep oasis/helpers/report_project.py, export writers, and dashboard indexing rules aligned when layout or metadata fields change.
• Canonical JSON analysis_root: New reports store the scanned tree relative to security_reports/; use oasis/helpers/analysis_root_path.py for resolution (integrations, assistant RAG cache root, scan_root candidates)—do not fork second resolution logic in web.py or the assistant.
• Package surface: Export the intended public helper API from oasis/helpers/__init__.py when symbols are meant for use outside the package; keep imports stable and explicit.
• Incremental scan progress (wire contract):
  • Constants (top of oasis/helpers/progress/__init__.py): EXEC_SUMMARY_PROGRESS_EVENT_VERSION bumps when the incremental progress payload is no longer backward-compatible for consumers (web.py, dashboard); extend SCAN_PROGRESS_EXTENDED_KEYS when new optional fields must pass through publish_incremental_summary and the executive-summary sidecar. Unknown keys are stripped there—keep the allowlist and Report._append_scan_progress_section aligned; contract coverage includes tests/test_report_schema.py (e.g. stripping unknown extras).
  • Timestamps: Use progress_timestamp_iso() in oasis/report.py for updated_at. Do not change the ISO-8601 shape without updating the dashboard stale guard in oasis/static/js/dashboard/api.js (lexicographic compare of UTC strings).
  • Sidecar: Persist incremental progress beside the executive summary JSON using executive_summary_progress_sidecar_path only—do not invent parallel path rules.
  • Helper roles: oasis/helpers/phases/scan.py (phase row builders and TypedDict wire shapes). oasis/helpers/progress/ (tqdm lifecycle, coercion, standard/adaptive extras, scan progress markdown helpers, LangGraph graph row builders). oasis/helpers/dashboard/ (parse_phase_counts_from_progress_cell for markdown table cells).
  • Audit markdown metrics contract: If oasis/report.py changes the ## Audit Metrics Summary markdown table (Metric | Value rows), keep oasis/helpers/dashboard/audit_metrics.py and oasis/web.py aggregation aligned so dashboard audit comparison remains backward-compatible.
  • Phase row status strings: Canonical values live in oasis/enums.py (PhaseRowStatus). Emit those wire strings from Python; the dashboard mirrors them as string constants only.
  • Graph pipeline rows: LangGraph runs use graph_progress_extras / graph_pipeline_phases in oasis/helpers/progress/ with ProgressPhaseRowId.GRAPH_DISCOVER … GRAPH_VERIFY. When changing phase ids, labels, or ordering, align progress/, any callers in analyze.py, and dashboard consumers if the wire shape changes.
• Favor small manager/service classes over monolithic procedural flows when adding features.
• Preserve CLI backward compatibility when possible (short and long flags); if a rename is required, mirror it in docs.
• For docs touched by behavior changes, keep README.md Features summary-only and place detailed behavior/usage in the relevant dedicated section (create one for new feature areas).
• Keep CHANGELOG.md entries concise, style-consistent, and filed under the version bucket that matches current branch lineage.
• Open-source hygiene: examples in code, help strings, docstrings, and tests use neutral placeholders (https://llm.example.com/v1) — never personal infrastructure details (hostnames, SSH aliases, keys, container names); such documentation lives in the uncommitted workspace mirror, not in the repo.
• Route logs through centralized project logging helpers; avoid ad-hoc print statements in production paths.
• For resilience changes (cache, network/model calls), prefer defensive fallbacks and explicit error handling.
• Tests: Integration-style checks live under tests/ as test_<area>.py files aligned with product modules (report contract in tests/test_report_schema.py, CLI in tests/test_oasis_cli.py, LangGraph orchestration in tests/test_analyze_orchestration.py, embedding pure helpers in tests/test_embedding_pure.py, dashboard helpers in tests/test_helpers_dashboard.py, model backends/providers in tests/test_backends_openai_compat.py, etc.). When you change behavior covered by those areas, update or add tests in the matching file rather than skipping coverage for new branches.