# Itemized Functions

> Generate exhaustive integration functions with comprehensive test suites for all 3rd-party APIs and external services. Automatically creates function wrappers, individual test files, integrated test runners, and a detailed report of API behavior, response signatures, latency, and failure modes.

- Skill: `harshitsinghbhandari/itemized-functions` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add harshitsinghbhandari/itemized-functions`
- Raw SKILL.md: https://api.skillmd.com/api/skills/harshitsinghbhandari/itemized-functions/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Docs & Writing
- Author: harshitsinghbhandari (https://skillmd.com/u/harshitsinghbhandari)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/harshitsinghbhandari/itemized-functions

---


## Overview

This skill analyzes architecture files to identify all external integrations (APIs, services, databases, etc.), then generates:

1. **Function wrappers** (`function_*.py`) — Clean, tested functions that use each integration exactly as specified in the architecture
2. **Individual test files** (`test_*.py`) — Comprehensive tests for each function covering success cases, failure modes, edge cases, and diverse input types
3. **Heavy API test suites** (when needed) — Separate test files for integrations requiring extensive data or API calls
4. **Integration test runner** (`run_all_tests.py`) — Master test orchestrator that runs all tests, collects results, and generates the final report
5. **Debug log** (`integrations.debug.log`) — Complete execution trace for troubleshooting
6. **Summary report** (`ITEMIZED_FUNCTIONS_REPORT.md`) — Detailed findings: function signatures, actual API responses (sanitized), latency metrics, test results, learnings

**Purpose:** Understand the *exact* behavior of every 3rd-party integration before building the full project. No assumptions. Real execution. Real data.

## Activation

**Trigger phrases:**
- "generate itemized functions"
- "create integration tests"
- "itemized functions from architecture"
- "test all integrations"

**Required input:** Architecture files must be provided or referenced. The skill reads the full architecture to understand all integrations, their purpose, and how they're used.

**Output:** All generated files are created in an `integration_tests/` directory at the repository root.

## Workflow

### Phase 1: Architecture Analysis

1. **Read all provided architecture files** (scan thoroughly, don't ask questions)
2. **Identify all external integrations**:
   - API services (Ollama, OpenAI, Anthropic, etc.)
   - Database systems (PostgreSQL, MongoDB, Redis, etc.)
   - File processing tools (GitHub Linguist, ImageMagick, etc.)
   - Message queues, cache systems, external webhooks, etc.
3. **For each integration, extract**:
   - Primary purpose (what the architecture says it's used for)
   - Specific use cases (e.g., "Ollama for tool calling" vs "Ollama for chat")
   - Expected inputs/outputs
   - Failure modes to test
   - Performance constraints or requirements
4. **Determine test complexity**:
   - Standard test: ≤10 API calls, <50MB data, simple responses
   - Heavy test: >10 API calls, >50MB data, streaming responses, or complex orchestration

### Phase 2: Credential Setup

Generate `.env.dev` at repository root with template entries for all required credentials:

```
# Ollama
OLLAMA_API_URL=http://localhost:11434
OLLAMA_MODEL=neural-chat

# GitHub Linguist
GITHUB_LINGUIST_PATH=/path/to/github-linguist

# PostgreSQL
DB_HOST=localhost
DB_PORT=5432
DB_NAME=testdb
DB_USER=testuser
DB_PASSWORD=

# [Other integrations...]
```

**Note:** User must fill in actual values before running tests.

### Phase 3: Generate Function Wrappers

For each integration, create `integration_tests/function_[service].py`:

**Requirements:**
- Clean, production-ready function signatures
- Proper type hints (Python 3.8+)
- Error handling with meaningful error messages
- Timeout handling (appropriate per service)
- Input validation where needed
- Logging to `integrations.debug.log`
- Return consistent, testable response objects

**Example structure:**
```python
import os
import logging
from typing import Any, Dict, List
import requests
from datetime import datetime

logger = logging.getLogger(__name__)

def call_ollama_chat(prompt: str, model: str = None, temperature: float = 0.7, timeout: int = 30) -> Dict[str, Any]:
    """
    Call Ollama API for chat completion.
    
    Args:
        prompt: The user prompt
        model: Model name (uses OLLAMA_MODEL env var if not provided)
        temperature: Sampling temperature (0.0-1.0)
        timeout: Request timeout in seconds
    
    Returns:
        Dict with keys: response, model, created_at, latency_ms
    
    Raises:
        ValueError: If credentials/config missing
        requests.Timeout: If request exceeds timeout
        requests.RequestException: For API errors
    """
    try:
        start_time = datetime.now()
        
        api_url = os.getenv("OLLAMA_API_URL", "http://localhost:11434")
        model = model or os.getenv("OLLAMA_MODEL")
        
        if not model:
            raise ValueError("OLLAMA_MODEL not set in environment")
        
        response = requests.post(
            f"{api_url}/api/chat",
            json={
                "model": model,
                "messages": [{"role": "user", "content": prompt}],
                "temperature": temperature,
                "stream": False
            },
            timeout=timeout
        )
        response.raise_for_status()
        
        latency_ms = (datetime.now() - start_time).total_seconds() * 1000
        result = response.json()
        result["latency_ms"] = latency_ms
        
        logger.debug(f"Ollama chat call successful. Latency: {latency_ms}ms")
        return result
        
    except requests.Timeout:
        logger.error(f"Ollama API timeout after {timeout}s")
        raise
    except requests.RequestException as e:
        logger.error(f"Ollama API error: {str(e)}")
        raise
    except Exception as e:
        logger.error(f"Unexpected error calling Ollama: {str(e)}")
        raise
```

### Phase 4: Generate Test Files

For each function, create `integration_tests/test_[service].py`:

**Requirements:**
- Use pytest framework
- Test success cases with typical inputs
- Test success cases with diverse/edge case inputs (smart per-service)
- Test failure modes: auth failure, timeout, malformed response, rate limiting, service down
- Capture actual API responses (sanitize credentials)
- Measure latency
- Each test logs to `integrations.debug.log`
- Each test validates response structure

**Example structure:**
```python
import pytest
import os
from unittest.mock import patch, MagicMock
import requests
from function_ollama import call_ollama_chat

@pytest.fixture
def setup_env(monkeypatch):
    """Setup environment variables for testing."""
    monkeypatch.setenv("OLLAMA_API_URL", "http://localhost:11434")
    monkeypatch.setenv("OLLAMA_MODEL", "neural-chat")

class TestOllamaChatSuccess:
    """Test successful Ollama chat calls."""
    
    def test_basic_chat(self, setup_env):
        """Test basic chat completion."""
        response = call_ollama_chat("What is 2+2?")
        assert "response" in response
        assert response["model"] == "neural-chat"
        assert "latency_ms" in response
        assert response["latency_ms"] > 0
    
    def test_chat_with_temperature(self, setup_env):
        """Test chat with different temperature values."""
        for temp in [0.0, 0.5, 1.0]:
            response = call_ollama_chat("Tell a story", temperature=temp)
            assert "response" in response
            assert response["latency_ms"] > 0
    
    def test_long_prompt(self, setup_env):
        """Test with very long prompt."""
        long_prompt = "What is the meaning of life? " * 100
        response = call_ollama_chat(long_prompt)
        assert "response" in response

class TestOllamaChatFailures:
    """Test failure modes."""
    
    def test_auth_failure(self, setup_env, monkeypatch):
        """Test behavior when API authentication fails."""
        monkeypatch.setenv("OLLAMA_API_URL", "http://invalid-url:11434")
        with pytest.raises(requests.RequestException):
            call_ollama_chat("test")
    
    def test_timeout(self, setup_env, monkeypatch):
        """Test timeout handling."""
        with patch('requests.post') as mock_post:
            mock_post.side_effect = requests.Timeout()
            with pytest.raises(requests.Timeout):
                call_ollama_chat("test", timeout=1)
    
    def test_missing_credentials(self, monkeypatch):
        """Test when required env vars are missing."""
        monkeypatch.delenv("OLLAMA_MODEL", raising=False)
        with pytest.raises(ValueError):
            call_ollama_chat("test")
```

### Phase 5: Heavy API Test Suites (When Applicable)

If an integration qualifies as "heavy" (>10 API calls, >50MB data, streaming, complex orchestration), create a separate `integration_tests/heavy_test_[service].py`:

**Include in heavy tests:**
- Large data processing (if applicable)
- Streaming response handling
- Multiple chained API calls
- Performance benchmarks
- Resource usage patterns
- Rate limiting behavior

**Header with reasoning:**
```python
"""
Heavy API test suite for [service].

Reasoning:
- [Service] requires extensive testing due to [specific reason]:
  - Streaming responses with large payloads
  - Multiple chained API calls (25+ total)
  - Data processing >50MB
  - Complex state management across calls
  - Critical performance path in architecture

These tests are separated from standard tests to avoid:
- Excessive API quota usage during CI/CD
- Extended test execution time
- Unnecessary load on rate-limited endpoints
"""
```

### Phase 6: Integration Test Runner

Create `integration_tests/run_all_tests.py`:

**Responsibilities:**
1. Import and run all test files using pytest
2. Collect results: passed, failed, skipped
3. For each test, capture:
   - Test name
   - Status (passed/failed/skipped)
   - Execution time
   - Error message (if failed)
   - Sample response data (if success)
4. Aggregate latency metrics per service
5. Generate summary statistics
6. Write all to `ITEMIZED_FUNCTIONS_REPORT.md`

**Example output structure:**
```
Test Results Summary:
- Total: 42 tests
- Passed: 38
- Failed: 2
- Skipped: 2

Service Latency Metrics:
- Ollama Chat: avg 145ms, min 89ms, max 287ms (10 calls)
- GitHub Linguist: avg 234ms, min 156ms, max 412ms (8 calls)
- PostgreSQL: avg 12ms, min 8ms, max 31ms (10 calls)

[Detailed results for each service...]
```

### Phase 7: Debug Logging

All generated code writes to `integration_tests/integrations.debug.log`:

- Timestamp, log level, service name, message
- Request/response bodies (sanitize credentials)
- Timing information
- Error stack traces
- Environment info (for debugging credential/setup issues)

Format:
```
[2024-01-15 14:32:15.342] DEBUG [ollama] Calling /api/chat with model=neural-chat
[2024-01-15 14:32:15.521] DEBUG [ollama] Response received: 145ms latency, 1250 chars
[2024-01-15 14:32:16.012] ERROR [github-linguist] FAILED_TO_TEST - Connection refused (auth_required, network_error, timeout, api_error, etc.)
```

### Phase 8: Summary Report

Create `ITEMIZED_FUNCTIONS_REPORT.md`:

**Structure:**
```markdown
# Itemized Functions Report

**Generated:** [timestamp]
**Architecture Analyzed:** [list of architecture files]
**Total Integrations Tested:** [count]
**Test Success Rate:** [X%]

## Executive Summary
- [count] integrations identified and tested
- [X] tests passed, [Y] failed, [Z] skipped
- Key findings and blockers (if any)

## Integration Details

### [Service Name] (e.g., Ollama)

**Purpose (from architecture):** [extracted from architecture]

**Function Signature:**
\`\`\`python
def call_ollama_chat(prompt: str, model: str = None, temperature: float = 0.7, timeout: int = 30) -> Dict[str, Any]
\`\`\`

**Test Coverage:** [count tests, all passed/mixed/failed]

**Latency:** avg X ms, min Y ms, max Z ms (10 calls)

**Sample API Response (sanitized):**
\`\`\`json
{
  "response": "2 + 2 = 4",
  "model": "neural-chat",
  "created_at": "2024-01-15T14:32:15Z",
  "latency_ms": 145
}
\`\`\`

**Failure Modes Tested:**
- ✓ Timeout (handled correctly, raises Timeout exception)
- ✓ Auth failure (handled correctly, raises RequestException)
- ✓ Malformed response (handled correctly, raises JSONDecodeError)
- ✓ Service unavailable (raises ConnectionError)

**Key Learnings:**
- [Finding 1]: [detail]
- [Finding 2]: [detail]
- [Gotcha/quirk if discovered]: [detail]

**Heavy Tests:** None
(or if applicable: `heavy_test_ollama.py` — [reason])

---

### [Next Service...]

[Same structure as above]

---

## Failed Tests & Blockers

### [Service Name] - FAILED_TO_TEST
**Reason:** [auth_required, network_error, timeout, api_error, service_down, etc.]
**Error Message:** [exact error]
**Suggestion:** [how to resolve, e.g., "Set OLLAMA_API_URL in .env.dev and ensure Ollama service is running"]

---

## Cross-Service Insights

[Any patterns, dependencies, or interactions discovered across integrations]

---

## Recommendations

- [Any critical issues or setup requirements]
- [Performance or scaling considerations]
- [Dependencies between services]

---

## Test Execution Log

[Link to or excerpt from integrations.debug.log]
```

## Standards & Requirements

### Code Generation

- **Python 3.8+ compatible** — Type hints, f-strings, async/await support if needed
- **Error messages are meaningful** — Not generic "error occurred", but "OLLAMA_API_URL not set" or "Connection refused on localhost:11434"
- **Timeouts are sensible per service** — LLM APIs: 60s, Database: 10s, File processing: 30s, etc.
- **No hardcoded values** — All config from environment
- **Logging is comprehensive** — Every significant action logged

### Testing

- **Pytest conventions** — `test_*.py` files, fixtures, clear test names, assertions with messages
- **Real execution** — Actually call the APIs (not mocked), unless explicitly impossible
- **Diverse inputs per service** — Smart, not generic:
  - LLM APIs: different prompt lengths, temperatures, contexts
  - File processing: different file types/sizes
  - Databases: different query patterns, edge cases
  - Time-series: different time ranges, aggregations
- **Failure mode coverage** — Auth, network, timeout, rate limit, malformed data
- **Response validation** — Verify structure, types, required fields
- **Latency tracking** — Measure every call

### Report Generation

- **No credentials exposed** — Sanitize all env vars, passwords, tokens, API keys in report
- **Full response bodies** — Show actual data (sanitized) so developer understands exact format
- **Timestamps throughout** — When was this run, when was each test executed
- **Actionable findings** — Not just "failed" but "why" and "how to fix"
- **Report all failures and anomalies** — Do not omit unexpected behavior because it seems minor. Every quirk discovered now prevents a production incident later
- **Output goes under the user's name** — The report becomes their reference document. Incomplete or softened findings waste the testing effort

## Special Cases

**Streaming APIs (e.g., LLM chat with stream=true):**
- Test both streamed and non-streamed responses
- Measure latency from first token to completion
- Validate chunk format

**Databases:**
- Test CRUD operations
- Test connection pooling if applicable
- Test transaction handling
- Test query timeouts

**File Processing APIs:**
- Test with multiple file types
- Test error handling for unsupported formats
- Test large file handling

**Rate-Limited APIs:**
- Detect rate limit headers
- Test behavior when rate limited
- Suggest backoff strategies in report

**Webhook/Async APIs:**
- If applicable, test callback handling
- Test idempotency if needed

## Directory Structure (Final)

```
integration_tests/
├── .env.dev                          # Template credentials file
├── integrations.debug.log            # Debug log from test execution
├── ITEMIZED_FUNCTIONS_REPORT.md      # Final summary report
├── run_all_tests.py                  # Master test runner
├── function_ollama.py                # Function wrapper
├── test_ollama.py                    # Standard tests
├── heavy_test_ollama.py              # (if needed) Heavy tests
├── function_github_linguist.py       # Another wrapper
├── test_github_linguist.py           # Standard tests
├── function_postgres.py              # Another wrapper
├── test_postgres.py                  # Standard tests
└── [More function/test pairs...]
```

## Execution

1. User provides architecture files and triggers skill
2. Skill generates all files in `integration_tests/`
3. User fills in `.env.dev` with actual credentials
4. User runs: `python run_all_tests.py`
5. All tests execute, report generated
6. Developer reviews `ITEMIZED_FUNCTIONS_REPORT.md` and `integrations.debug.log`

## Quality Assurance

- **No questions asked** — Read architecture, infer integrations, generate tests
- **LLM confidence in test design** — Trust that generated tests cover the right scenarios
- **Sanitization is rigorous** — Scan all report content for credentials before writing
- **File encoding is UTF-8** — Handle responses with special characters correctly
- **All files importable** — Generated Python is syntactically correct and runs