modbender X Bookmark Archiver
Archive your X (Twitter) bookmarks into categorized markdown files with AI-generated summaries.
Overview
This skill fetches your X bookmarks using the bird CLI, categorizes them by URL patterns, generates AI summaries using OpenAI, and saves them as organized markdown files.
Prerequisites
- bird CLI - Install from steipete/bird
- OpenAI API Key (optional) - Set
OPENAI_API_KEYfor AI-generated summaries - Node.js 18+
Installation
# Ensure bird CLI is installed and authenticated
bird --version
# Set OpenAI API key (optional, for AI summaries)
export OPENAI_API_KEY="sk-..."
Commands
run - Full Pipeline
Fetches new bookmarks and processes them:
node skills/x-bookmark-archiver/scripts/run.cjs
run --force - Process Existing
Skip fetch, process only pending bookmarks:
node skills/x-bookmark-archiver/scripts/run.cjs --force
fetch - Download Bookmarks Only
node skills/x-bookmark-archiver/scripts/fetch.cjs
process - Archive Pending Only
node skills/x-bookmark-archiver/scripts/process.cjs
Category Mapping
Bookmarks are automatically categorized based on URL patterns:
| Category | Domains |
|---|---|
| tools | github.com, gitlab.com, github.io, huggingface.co, replicate.com, vercel.com, npmjs.com, pypi.org |
| articles | medium.com, substack.com, dev.to, hashnode.dev, x.com/i/article, blog.*, towardsdatascience.com |
| videos | youtube.com, youtu.be, vimeo.com, twitch.tv |
| research | arxiv.org, paperswithcode.com, semanticscholar.org, researchgate.net, dl.acm.org, ieee.org |
| news | techcrunch.com, theverge.com, hn.algolia.com, news.ycombinator.com, wired.com, arstechnica.com |
| bookmarks | fallback for unmatched URLs |
Output Location
Markdown files are created in the OpenClaw workspace at:
Legacy installs (old name):
~/clawd/X-knowledge/
New installs:
~/.openclaw/workspace/X-knowledge/
With profile (OPENCLAW_PROFILE=prod):
~/.openclaw/workspace-prod/X-knowledge/
Override with environment variable:
export OPENCLAW_WORKSPACE=/custom/path
node skills/x-bookmark-archiver/scripts/run.cjs
# Creates: /custom/path/X-knowledge/
Output Structure
~/.openclaw/workspace/X-knowledge/
├── tools/
│ ├── awesome-ai-project.md
│ └── useful-cli-tool.md
├── articles/
│ ├── how-to-build-x.md
│ └── ml-best-practices.md
├── videos/
│ └── conference-talk.md
├── research/
│ └── attention-is-all-you-need.md
├── news/
│ └── latest-tech-announcement.md
└── bookmarks/
└── misc-link.md
Markdown Template
Each archived bookmark gets a markdown file with frontmatter:
---
title: "Awesome AI Project"
type: tool
date_archived: 2026-01-31
source_tweet: https://x.com/i/web/status/1234567890
link: https://github.com/user/repo
tags: ["ai", "machine-learning", "github"]
---
This project implements a novel approach to... (AI-generated summary)
State Management
State files track processing progress:
/root/clawd/.state/
├── x-bookmark-pending.json # Bookmarks waiting to be processed
└── x-bookmark-processed.json # IDs of already-archived bookmarks
Environment Variables
| Variable | Required | Description |
|---|---|---|
OPENAI_API_KEY |
No | API key for AI-generated summaries |
Workflow
- Fetch: Downloads latest 50 bookmarks from X
- Filter: Removes already-processed bookmarks
- Expand: Resolves t.co shortened URLs
- Categorize: Assigns category based on URL domain
- Enrich: Generates title, summary, tags (AI or fallback)
- Write: Saves as markdown in
X-knowledge/{category}/ - Track: Updates processed IDs, clears pending
Customization
Adding Categories
Edit scripts/lib/categorize.cjs:
const CATEGORIES = {
tools: ['github.com', '...'],
your_category: ['example.com', '...'],
// ...
};
Changing Output Directory
Edit scripts/process.cjs:
const KNOWLEDGE_DIR = 'your-directory-name';
Using Different AI Provider
Modify the generateMetadata() function in scripts/process.cjs to use your preferred API.
Testing
Run the test suite:
# Run all tests
cd skills/x-bookmark-archiver/tests
node test-all.cjs
# Run individual test suites
node lib/categorize.test.cjs
node lib/state.test.cjs
node integration.test.cjs
Test Coverage
- Unit tests:
categorize.js(21 tests) - URL pattern matching - Unit tests:
state.js(9 tests) - JSON read/write operations - Integration tests (12 tests) - Full pipeline with mock data
Manual Testing
Without bird CLI, you can test with mock data:
# Create mock pending data
cat > /tmp/test-pending.json << 'EOF'
[
{
"id": "test123",
"url": "https://github.com/test/repo",
"text": "Test bookmark"
}
]
EOF
# Copy to state directory and process
mkdir -p /root/clawd/.state
cp /tmp/test-pending.json /root/clawd/.state/x-bookmark-pending.json
node skills/x-bookmark-archiver/scripts/process.cjs
Troubleshooting
| Issue | Solution |
|---|---|
bird: command not found |
Install bird CLI from GitHub releases |
| No bookmarks fetched | Ensure you're logged into X in bird |
| AI summaries not generating | Check OPENAI_API_KEY is set |
| t.co links not expanding | May be network/timeout issues; will use original URL |
File Structure
skills/x-bookmark-archiver/
├── SKILL.md
├── scripts/
│ ├── fetch.cjs # Download bookmarks from X (CommonJS)
│ ├── process.cjs # Generate markdown files (CommonJS)
│ ├── run.cjs # Orchestrate fetch → process (CommonJS)
│ └── lib/
│ ├── categorize.cjs # URL → category mapping (CommonJS)
│ └── state.cjs # JSON state management (CommonJS)
└── tests/
├── test-all.cjs
├── lib/
│ ├── categorize.test.cjs
│ │ └── state.test.cjs
├── integration.test.cjs
└── fixtures/
└── sample-bookmarks.json
License
MIT