SourceMonitor Domain Model
Overview
SourceMonitor is a Rails 8 mountable engine for RSS/feed monitoring. All models live under the SourceMonitor:: namespace and inherit from SourceMonitor::ApplicationRecord. Tables use a configurable prefix (default: sourcemon_).
Model Graph
Source (central entity)
|-- has_many :items (active only, via scope)
|-- has_many :all_items (includes soft-deleted)
|-- has_many :fetch_logs
|-- has_many :scrape_logs
|-- has_many :health_check_logs
|-- has_many :log_entries
|
Item
|-- belongs_to :source (counter_cache: true)
|-- has_one :item_content (dependent: :destroy, autosave: true)
|-- has_many :scrape_logs
|-- has_many :log_entries
|
ItemContent
|-- belongs_to :item (touch: true)
|
FetchLog (includes Loggable)
|-- belongs_to :source
|-- has_one :log_entry (as: :loggable, polymorphic)
|
ScrapeLog (includes Loggable)
|-- belongs_to :item
|-- belongs_to :source
|-- has_one :log_entry (as: :loggable, polymorphic)
|
HealthCheckLog (includes Loggable)
|-- belongs_to :source
|-- has_one :log_entry (as: :loggable, polymorphic)
|
LogEntry (delegated_type :loggable)
|-- belongs_to :source
|-- belongs_to :item (optional)
|-- loggable types: FetchLog, ScrapeLog, HealthCheckLog
|
ImportSession (standalone)
|-- user_id (FK to host app)
|
ImportHistory (standalone)
|-- user_id (FK to host app)
Models Summary
| Model |
Table |
Purpose |
Source |
sourcemon_sources |
Feed source with URL, fetch config, health tracking |
Item |
sourcemon_items |
Individual feed entry/article |
ItemContent |
sourcemon_item_contents |
Scraped HTML/content (split from items for performance) |
FetchLog |
sourcemon_fetch_logs |
Record of each feed fetch attempt |
ScrapeLog |
sourcemon_scrape_logs |
Record of each item scrape attempt |
HealthCheckLog |
sourcemon_health_check_logs |
Record of each health check |
LogEntry |
sourcemon_log_entries |
Unified log view via delegated_type (polymorphic) |
ImportSession |
sourcemon_import_sessions |
OPML import wizard state |
ImportHistory |
sourcemon_import_histories |
Completed import records |
Key Concerns
Loggable (app/models/concerns/source_monitor/loggable.rb)
Shared by FetchLog, ScrapeLog, HealthCheckLog:
attribute :metadata, default: -> { {} }validates :started_at, presence: truevalidates :duration_ms, numericality: { >= 0 }, allow_nil: true- Scopes:
recent, successful, failed
Sanitizable (lib/source_monitor/models/sanitizable.rb)
String/hash attribute sanitization. Used by Source.
UrlNormalizable (lib/source_monitor/models/url_normalizable.rb)
URL normalization and format validation. Used by Source and Item.
Source Model Details
State Values
| Field |
Values |
Notes |
fetch_status |
idle, queued, fetching, failed, invalid |
DB CHECK constraint |
health_status |
working (default) |
Values: working, declining, improving, failing |
active |
true/false |
Boolean toggle |
Key Scopes
| Scope |
Meaning |
active |
WHERE active = true |
failed |
failure_count > 0 OR last_error IS NOT NULL OR last_error_at IS NOT NULL |
healthy |
active AND failure_count = 0 AND last_error IS NULL AND last_error_at IS NULL (Note: scope name preserved for backward compat; health_status uses "working") |
due_for_fetch(reference_time:) |
Class method. Active sources where next_fetch_at IS NULL OR <= reference_time |
Validations
| Field |
Rules |
name |
presence |
feed_url |
presence, uniqueness (case insensitive) |
fetch_interval_minutes |
numericality > 0 |
scraper_adapter |
presence |
items_retention_days |
integer >= 0, allow nil |
max_items |
integer >= 0, allow nil |
fetch_status |
inclusion in FETCH_STATUS_VALUES |
fetch_retry_attempt |
integer >= 0 |
health_auto_pause_threshold |
custom: 0..1 range |
Notable Methods
fetch_interval_hours / fetch_interval_hours= -- convenience accessors converting minutesfetch_circuit_open? -- circuit breaker checkauto_paused? -- health-based auto-pause checkreset_items_counter! -- recalculate counter cache from active items
Item Model Details
Soft Delete Pattern
Items use soft delete via deleted_at column (NOT default_scope):
scope :active -- WHERE deleted_at IS NULLscope :with_deleted -- unscopes deleted_atscope :only_deleted -- WHERE deleted_at IS NOT NULLsoft_delete!(timestamp:) -- sets deleted_at, decrements counter cachedeleted? -- checks deleted_at presence
Key Scopes
| Scope |
Meaning |
active |
WHERE deleted_at IS NULL |
recent |
Active, ordered by published_at DESC NULLS LAST, created_at DESC |
published |
Active, WHERE published_at IS NOT NULL |
pending_scrape |
Active, WHERE scraped_at IS NULL |
failed_scrape |
Active, WHERE scrape_status = 'failed' |
Validations
| Field |
Rules |
source |
presence |
guid |
presence, uniqueness scoped to source_id (case insensitive) |
content_fingerprint |
uniqueness scoped to source_id, allow blank |
url |
presence |
Content Delegation
scraped_html and scraped_content delegate to ItemContent. Setting these values auto-creates the ItemContent association. ensure_feed_content_record also creates ItemContent for feed word count tracking. ItemContent is only auto-destroyed when both scraped fields are blank AND the item has no feed content.
LogEntry Delegated Type
LogEntry uses Rails delegated_type to unify FetchLog, ScrapeLog, and HealthCheckLog:
loggable_type -- polymorphic type columnloggable_id -- polymorphic ID column- Helper methods:
fetch?, scrape?, health_check?, log_type
- After-save sync: each log type calls
Logs::EntrySync.call(self) to keep LogEntry in sync
ImportSession Wizard Steps
Ordered steps: upload -> preview -> health_check -> configure -> confirm
Methods: next_step, previous_step, health_stream_name, health_check_targets
ModelExtensions Registry
All models register via SourceMonitor::ModelExtensions.register(self, :key). This system:
- Assigns table names using the configured prefix
- Applies host-app concerns
- Applies host-app validations
- Supports
reload! for configuration changes
References
- Model Relationship Graph -- Visual model relationships
- Table Structure -- Complete schema with columns, types, indexes
- Source files:
app/models/source_monitor/*.rb
- Concern:
app/models/concerns/source_monitor/loggable.rb
- Migrations:
db/migrate/*.rb
Converted and distributed by TomeVault — claim your Tome and manage your conversions.
1---2name: sm-domain-model3description: Provides SourceMonitor engine domain model context. Use when working with engine models, associations, validations, scopes, database schema, or any model-layer code in the source_monitor namespace.4---56# SourceMonitor Domain Model78## Overview910SourceMonitor is a Rails 8 mountable engine for RSS/feed monitoring. All models live under the `SourceMonitor::` namespace and inherit from `SourceMonitor::ApplicationRecord`. Tables use a configurable prefix (default: `sourcemon_`).1112## Model Graph1314```15Source (central entity)16 |-- has_many :items (active only, via scope)17 |-- has_many :all_items (includes soft-deleted)18 |-- has_many :fetch_logs19 |-- has_many :scrape_logs20 |-- has_many :health_check_logs21 |-- has_many :log_entries22 |23 Item24 |-- belongs_to :source (counter_cache: true)25 |-- has_one :item_content (dependent: :destroy, autosave: true)26 |-- has_many :scrape_logs27 |-- has_many :log_entries28 |29 ItemContent30 |-- belongs_to :item (touch: true)31 |32 FetchLog (includes Loggable)33 |-- belongs_to :source34 |-- has_one :log_entry (as: :loggable, polymorphic)35 |36 ScrapeLog (includes Loggable)37 |-- belongs_to :item38 |-- belongs_to :source39 |-- has_one :log_entry (as: :loggable, polymorphic)40 |41 HealthCheckLog (includes Loggable)42 |-- belongs_to :source43 |-- has_one :log_entry (as: :loggable, polymorphic)44 |45 LogEntry (delegated_type :loggable)46 |-- belongs_to :source47 |-- belongs_to :item (optional)48 |-- loggable types: FetchLog, ScrapeLog, HealthCheckLog49 |50 ImportSession (standalone)51 |-- user_id (FK to host app)52 |53 ImportHistory (standalone)54 |-- user_id (FK to host app)55```5657## Models Summary5859| Model | Table | Purpose |60|-------|-------|---------|61| `Source` | `sourcemon_sources` | Feed source with URL, fetch config, health tracking |62| `Item` | `sourcemon_items` | Individual feed entry/article |63| `ItemContent` | `sourcemon_item_contents` | Scraped HTML/content (split from items for performance) |64| `FetchLog` | `sourcemon_fetch_logs` | Record of each feed fetch attempt |65| `ScrapeLog` | `sourcemon_scrape_logs` | Record of each item scrape attempt |66| `HealthCheckLog` | `sourcemon_health_check_logs` | Record of each health check |67| `LogEntry` | `sourcemon_log_entries` | Unified log view via delegated_type (polymorphic) |68| `ImportSession` | `sourcemon_import_sessions` | OPML import wizard state |69| `ImportHistory` | `sourcemon_import_histories` | Completed import records |7071## Key Concerns7273### Loggable (`app/models/concerns/source_monitor/loggable.rb`)74Shared by FetchLog, ScrapeLog, HealthCheckLog:75- `attribute :metadata, default: -> { {} }`76- `validates :started_at, presence: true`77- `validates :duration_ms, numericality: { >= 0 }, allow_nil: true`78- Scopes: `recent`, `successful`, `failed`7980### Sanitizable (`lib/source_monitor/models/sanitizable.rb`)81String/hash attribute sanitization. Used by Source.8283### UrlNormalizable (`lib/source_monitor/models/url_normalizable.rb`)84URL normalization and format validation. Used by Source and Item.8586## Source Model Details8788### State Values8990| Field | Values | Notes |91|-------|--------|-------|92| `fetch_status` | `idle`, `queued`, `fetching`, `failed`, `invalid` | DB CHECK constraint |93| `health_status` | `working` (default) | Values: working, declining, improving, failing |94| `active` | `true`/`false` | Boolean toggle |9596### Key Scopes9798| Scope | Meaning |99|-------|---------|100| `active` | `WHERE active = true` |101| `failed` | `failure_count > 0 OR last_error IS NOT NULL OR last_error_at IS NOT NULL` |102| `healthy` | `active AND failure_count = 0 AND last_error IS NULL AND last_error_at IS NULL` (Note: scope name preserved for backward compat; health_status uses "working") |103| `due_for_fetch(reference_time:)` | Class method. Active sources where `next_fetch_at IS NULL OR <= reference_time` |104105### Validations106107| Field | Rules |108|-------|-------|109| `name` | presence |110| `feed_url` | presence, uniqueness (case insensitive) |111| `fetch_interval_minutes` | numericality > 0 |112| `scraper_adapter` | presence |113| `items_retention_days` | integer >= 0, allow nil |114| `max_items` | integer >= 0, allow nil |115| `fetch_status` | inclusion in FETCH_STATUS_VALUES |116| `fetch_retry_attempt` | integer >= 0 |117| `health_auto_pause_threshold` | custom: 0..1 range |118119### Notable Methods120121- `fetch_interval_hours` / `fetch_interval_hours=` -- convenience accessors converting minutes122- `fetch_circuit_open?` -- circuit breaker check123- `auto_paused?` -- health-based auto-pause check124- `reset_items_counter!` -- recalculate counter cache from active items125126## Item Model Details127128### Soft Delete Pattern129Items use soft delete via `deleted_at` column (NOT default_scope):130- `scope :active` -- `WHERE deleted_at IS NULL`131- `scope :with_deleted` -- unscopes deleted_at132- `scope :only_deleted` -- `WHERE deleted_at IS NOT NULL`133- `soft_delete!(timestamp:)` -- sets deleted_at, decrements counter cache134- `deleted?` -- checks deleted_at presence135136### Key Scopes137138| Scope | Meaning |139|-------|---------|140| `active` | `WHERE deleted_at IS NULL` |141| `recent` | Active, ordered by `published_at DESC NULLS LAST, created_at DESC` |142| `published` | Active, `WHERE published_at IS NOT NULL` |143| `pending_scrape` | Active, `WHERE scraped_at IS NULL` |144| `failed_scrape` | Active, `WHERE scrape_status = 'failed'` |145146### Validations147148| Field | Rules |149|-------|-------|150| `source` | presence |151| `guid` | presence, uniqueness scoped to source_id (case insensitive) |152| `content_fingerprint` | uniqueness scoped to source_id, allow blank |153| `url` | presence |154155### Content Delegation156`scraped_html` and `scraped_content` delegate to `ItemContent`. Setting these values auto-creates the ItemContent association. `ensure_feed_content_record` also creates ItemContent for feed word count tracking. ItemContent is only auto-destroyed when both scraped fields are blank AND the item has no feed content.157158## LogEntry Delegated Type159160LogEntry uses Rails `delegated_type` to unify FetchLog, ScrapeLog, and HealthCheckLog:161- `loggable_type` -- polymorphic type column162- `loggable_id` -- polymorphic ID column163- Helper methods: `fetch?`, `scrape?`, `health_check?`, `log_type`164- After-save sync: each log type calls `Logs::EntrySync.call(self)` to keep LogEntry in sync165166## ImportSession Wizard Steps167168Ordered steps: `upload` -> `preview` -> `health_check` -> `configure` -> `confirm`169170Methods: `next_step`, `previous_step`, `health_stream_name`, `health_check_targets`171172## ModelExtensions Registry173174All models register via `SourceMonitor::ModelExtensions.register(self, :key)`. This system:1751. Assigns table names using the configured prefix1762. Applies host-app concerns1773. Applies host-app validations1784. Supports `reload!` for configuration changes179180## References181182- [Model Relationship Graph](reference/model-graph.md) -- Visual model relationships183- [Table Structure](reference/table-structure.md) -- Complete schema with columns, types, indexes184- Source files: `app/models/source_monitor/*.rb`185- Concern: `app/models/concerns/source_monitor/loggable.rb`186- Migrations: `db/migrate/*.rb`187188---189> Converted and distributed by [TomeVault](https://tomevault.io/claim/dchuk) — claim your Tome and manage your conversions.190<!-- tomevault:4.0:skill_md:2026-04-15 -->
tomevault-io