# Review Reply Suggestion Feedback

> Daily scheduled skill that reviews yesterday's #feed-mentions and #community-mentions activity, compares Buzz's reply suggestions to the team's emoji reactions and thread feedback, and opens one PR to improve draft-brand-reply if patterns emerge.

- Skill: `warpdotdev/review-reply-suggestion-feedback` (Agent Skill)
- Install (CLI): `npx skillmds@latest add warpdotdev/review-reply-suggestion-feedback`
- Raw SKILL.md: https://api.skillmd.com/api/skills/warpdotdev/review-reply-suggestion-feedback/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: warpdotdev (https://skillmd.com/u/warpdotdev)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/warpdotdev/review-reply-suggestion-feedback

---


# Review Reply Suggestion Feedback

You are a daily self-improvement loop for Buzz's community reply quality. Each morning you review yesterday's `#feed-mentions` and `#community-mentions` activity, compare Buzz's suggestions to what the team actually did, and — if there are durable patterns — update the `draft-brand-reply` skill and open one PR.

## Inputs

Your prompt may optionally contain:
- **Date to review** — a specific date (YYYY-MM-DD) to analyze

If no date is provided, default to **yesterday** (UTC).

## Channels

Review both channels in one run:

- `#feed-mentions`: `FEED_MENTIONS_CHANNEL_ID`
- `#community-mentions`: `COMMUNITY_MENTIONS_CHANNEL_ID`

Use `$BUZZ_SLACK_TOKEN`, `$FEED_MENTIONS_CHANNEL_ID`, and `$COMMUNITY_MENTIONS_CHANNEL_ID` via `os.environ` for Slack reads and status updates. Never echo, log, or hardcode it.

## Process

### 1. Collect raw data from Slack

Write and run a Python script that gathers yesterday's parent messages, reactions, and thread replies for both channels. The script collects **raw data only** — it does not try to parse Buzz's suggestion or source message body from the message format. You'll interpret that yourself in Step 2.

Use this shape for the collection script:

```python
import json, os, urllib.parse, urllib.request
from datetime import datetime, timezone, timedelta

TOKEN = os.environ["BUZZ_SLACK_TOKEN"]
REVIEW_DATE = os.environ.get("REVIEW_DATE") or (datetime.now(timezone.utc) - timedelta(days=1)).strftime("%Y-%m-%d")
CHANNELS = [
    {"name": "#feed-mentions", "id": os.environ.get("FEED_MENTIONS_CHANNEL_ID", "")},
    {"name": "#community-mentions", "id": os.environ.get("COMMUNITY_MENTIONS_CHANNEL_ID", "")},
]
CHANNELS = [channel for channel in CHANNELS if channel["id"]]

def slack_api(method, params=None):
    url = f"https://slack.com/api/{method}"
    if params:
        url += "?" + urllib.parse.urlencode(params)
    req = urllib.request.Request(url, headers={"Authorization": f"Bearer {TOKEN}"})
    return json.loads(urllib.request.urlopen(req).read())

day_start = datetime.strptime(REVIEW_DATE, "%Y-%m-%d").replace(tzinfo=timezone.utc)
day_end = day_start + timedelta(days=1)
oldest = str(int(day_start.timestamp()))
latest = str(int(day_end.timestamp()))

results = []
for channel in CHANNELS:
    history = slack_api("conversations.history", {
        "channel": channel["id"], "oldest": oldest, "latest": latest, "limit": "200"
    })
    for msg in history.get("messages", []):
        ts = msg["ts"]
        msg_text = msg.get("text", "")
        if "⏭️" in msg_text and "Skipped mentions" in msg_text:
            continue
        if msg_text.startswith("+") and "more unanswered community messages" in msg_text:
            continue

        reactions_resp = slack_api("reactions.get", {"channel": channel["id"], "timestamp": ts})
        reaction_list = reactions_resp.get("message", {}).get("reactions", [])
        reactions = {r["name"]: r["count"] for r in reaction_list}

        team_decision = None
        if "white_check_mark" in reactions:
            team_decision = "reply"
        elif "heart" in reactions:
            team_decision = "like"
        elif "x" in reactions:
            team_decision = "skip"

        thread = slack_api("conversations.replies", {
            "channel": channel["id"], "ts": ts, "limit": "50"
        })
        replies = thread.get("messages", [])[1:]

        human_feedback = []
        for reply in replies:
            if reply.get("bot_id") or reply.get("subtype") == "bot_message":
                continue
            if reply.get("ts") == ts:
                continue
            human_feedback.append(reply.get("text", ""))

        if team_decision is None and not human_feedback:
            continue

        results.append({
            "channel_name": channel["name"],
            "channel_id": channel["id"],
            "ts": ts,
            "message_text": msg_text,
            "blocks": msg.get("blocks", []),
            "team_decision": team_decision,
            "human_feedback": human_feedback,
        })

print(json.dumps(results, indent=2))
```

If the prompt specifies a date, set `REVIEW_DATE` as an env var before running, or let it default to yesterday.

### 2. Interpret the collected data

For each collected message, read `message_text` and `blocks` to determine:

- **Channel** — whether this came from `#feed-mentions` or `#community-mentions`.
- **Buzz's suggestion** — look for the triage indicator in the message (for example ✅ suggested reply, ❤️ suggestion: like, ❌ suggestion: skip). Community cards usually only have suggested replies.
- **Buzz's draft** — if a reply was suggested, extract the draft text from the message.
- **Source text** — the quoted social/community message excerpt from the card.
- **Mismatch** — whether Buzz's suggestion differs from the team's reaction (`team_decision`).

Don't rely on exact formatting — use your judgment to identify these elements from message content and blocks. The format may evolve over time.

### 3. Identify cases worth learning from

From your interpretation in Step 2, filter to cases that have learning value:

- **Reaction mismatches** — Buzz suggested X, team reacted Y. If the reason for the mismatch is obvious from context, include it. If it's ambiguous *and* there's no thread feedback, ignore it.
- **Thread feedback** — any thread where a human posted a message. Include these regardless of whether the reaction matched.
- **No Buzz suggestion** — Buzz didn't post a suggestion but the team did react or leave feedback. The team's reaction tells us the right call even without Buzz's take.
- **No signal** — no reaction and no feedback. Skip entirely.

If there are zero cases worth learning from across both channels, skip to step 6 (status update).

### 4. Learn from the cases

Use the `reply-learning` skill to analyze the combined batch of cases from both channels. Present them clearly:

For each case, provide:
- The channel
- The original social/community text
- Buzz's suggestion (or "no suggestion") and draft if applicable
- The team's reaction
- Any human feedback messages
- Whether it was a match, mismatch, or missing suggestion

The `reply-learning` skill will determine whether the cases reveal durable patterns worth codifying in `draft-brand-reply/SKILL.md`. Follow its process — it may conclude that no changes are needed even with mismatches (one-offs, ambiguous cases, etc.).

### 5. Open a PR if changes were made

If `draft-brand-reply/SKILL.md` was edited:

1. Create a branch: `buzz/reply-learning-YYYY-MM-DD`
2. Commit the changes with a message explaining what was learned
3. Open a **ready-for-review** PR (not a draft) using `gh pr create` — do NOT pass `--draft`:
   - Title: `draft-brand-reply: learnings from YYYY-MM-DD feedback`
   - Body: summary of what was learned, the specific cases that drove the changes, and the diff
   - CODEOWNERS will auto-assign reviewers

If no edits were warranted, skip the PR.

### 6. Post status updates to Slack

Post one status message to each reviewed channel using `BUZZ_SLACK_TOKEN`.

**If a PR was opened:**
> 📝 *Daily reply review (YYYY-MM-DD)*
> Reviewed N suggestions in this channel. Found X mismatches and Y feedback threads.
> Opened a PR with improvements: <PR_URL|link>
> Brief summary of what changed.

**If no PR was opened:**
> 📝 *Daily reply review (YYYY-MM-DD)*
> Reviewed N suggestions in this channel. [Reason no PR was needed — e.g. "All suggestions matched team decisions" or "2 mismatches found but no durable patterns to codify"]

Use the Buzz bot identity. Include `unfurl_links: False` and `unfurl_media: False`. Keep the status concise; one top-level message per channel, no threads.

## Constraints

- Do NOT fabricate feedback or reactions — only use what's actually in Slack
- Do NOT make changes to `draft-brand-reply` unless the `reply-learning` process concludes they're warranted
- Do NOT include the Slack token in any output, log, or file
- Do NOT post the status update until all other steps are complete
- Keep the status update concise — one message, no threads