Synthesize User Feedback from 5 Channels with AI CLI Agents

What You Will Build

A multi-agent pipeline that reads user feedback from five channels -- GitHub Issues, Slack, Linear, email exports, and app store reviews -- classifies each piece by sentiment and topic, clusters them into themes, and produces a prioritized insight report. You run one command: claude "synthesize this week's feedback". Within 60 seconds, you get a structured report showing which themes are trending, which are urgent, and which have the most evidence across channels. Every insight includes source quotes so you can verify without digging through raw data.

By the end of this article, you will have:

• Five MCP server connections pulling from your actual feedback channels
• An orchestrator-workers agent pattern: one worker per channel, one synthesizer that merges results
• Structured output with sentiment scores, confidence levels, and evidence quotes
• A CLAUDE.md file encoding your product domain knowledge for accurate classification
• Batch processing configuration that cuts API costs by roughly 50%
• Prompt caching for recurring weekly runs

The total setup takes about 45 minutes. After that, weekly feedback synthesis drops from a full PM workday to a single terminal command.

The Problem: Feedback Scattered Across 5 Places

If you ship a developer tool -- or any SaaS product -- user feedback arrives everywhere:

• GitHub Issues: bug reports, feature requests, sometimes disguised as questions
• Slack: community channels, support threads, casual mentions
• Linear: internal bug triage, customer-reported tickets routed by support
• Email: support inbox exports, sales-forwarded feedback, direct user emails
• App store reviews: if you have a desktop or mobile app, star ratings with text

A PM doing this manually opens five tabs, reads through dozens of items, mentally tags each one, tries to spot patterns, and writes up a summary. This takes 4-6 hours per week. Themes get missed because human working memory caps out after 30-40 items. Urgent issues buried in a Slack thread at 2 AM get noticed three days late.

The data is already structured enough for machine reading. GitHub Issues have labels, timestamps, and reaction counts. Slack messages have channels, threads, and emoji reactions. Linear tickets have priorities and statuses. The missing piece is aggregation and pattern detection across all five sources simultaneously.

Architecture: Orchestrator-Workers Pattern

This pipeline uses a multi-agent pattern. Instead of one agent sequentially processing all five channels (slow, context-heavy), you run one worker agent per channel in parallel, then a synthesizer agent that merges the results.

                    Orchestrator
                   /  |   |  |  \
                  /   |   |  |   \
           GitHub  Slack Linear Email  Reviews
           Worker  Worker Worker Worker Worker
                  \   |   |  |   /
                   \  |   |  |  /
                    Synthesizer
                        |
                  Insight Report

Why this pattern matters:

1. Parallelism. Five workers run simultaneously. A sequential pipeline takes 5x longer.
2. Isolation. Each worker's context window only contains data from its channel. No cross-contamination, no context overflow.
3. Specialization. Each worker's prompt is tuned for its data format. The GitHub worker knows how to parse issue labels. The Slack worker knows how to extract signal from noisy threads.
4. Cost efficiency. Workers process raw data into compact summaries. The synthesizer only sees pre-processed, structured output -- far fewer tokens than the raw input.

Prerequisites

You need these installed and authenticated:

• Claude Code v2.1+ with API access
• Node.js 18+ (for MCP servers)
• GitHub CLI (gh) authenticated with your account
• A Slack workspace with a bot token
• A Linear account with an API key
• Email feedback exported as .csv or .json (from your support tool -- Zendesk, Intercom, or a simple mail export)
• App store reviews exported as .csv (from App Store Connect, Google Play Console, or a scraping tool)

If you have never set up MCP servers, the Build Your First MCP Server tutorial covers the fundamentals.

Step 1: Configure MCP Server Connections

Set up connections for the three API-based channels. Email and app store reviews use local file reads -- no MCP server needed for those.

{
  "mcpServers": {
    "github": {
      "command": "npx",
      "args": ["-y", "@modelcontextprotocol/server-github"],
      "env": {
        "GITHUB_PERSONAL_ACCESS_TOKEN": "ghp_your_token_here"
      }
    },
    "slack": {
      "command": "npx",
      "args": ["-y", "@anthropic/mcp-server-slack"],
      "env": {
        "SLACK_BOT_TOKEN": "xoxb-your-bot-token",
        "SLACK_TEAM_ID": "T01ABCDEF"
      }
    },
    "linear": {
      "command": "npx",
      "args": ["-y", "@linear/mcp-server"],
      "env": {
        "LINEAR_API_KEY": "lin_api_your_key_here"
      }
    }
  }
}

For email and app store reviews, export your data weekly as files into a feedback/ directory:

feedback/
  email-export-2026-03-22.csv
  appstore-reviews-2026-03-22.csv

The agent reads these with standard file tools. No MCP server required. This keeps the setup simple and avoids building custom integrations for data sources that already have export functionality.

Important: Store tokens in environment variables, not in .mcp.json directly. Use a .env file or your shell profile.

Step 2: Encode Domain Knowledge in CLAUDE.md

The difference between useful feedback classification and noise is domain knowledge. A generic sentiment classifier does not know that "the CLI hangs when I pipe output" is a P0 bug for a developer tool, while "would be nice to have dark mode" is a P3 feature request.

Add this to your project's CLAUDE.md:

## Feedback Synthesis Workflow

### Product Context
Product: [Your Product Name] -- a developer CLI tool for [description].
Key user personas: individual developers, tech leads, DevOps engineers.
Current version: v2.3. Recent major release: v2.0 (new plugin system).

### Classification Taxonomy
When classifying feedback, use these categories:
- **bug-critical**: crashes, data loss, security issues, blocking workflows
- **bug-minor**: UI glitches, typos, non-blocking annoyances
- **feature-request**: new capabilities not currently in the product
- **ux-improvement**: existing features that work but are hard to use
- **documentation**: missing docs, unclear instructions, outdated examples
- **praise**: positive feedback (track for morale and marketing)
- **noise**: off-topic, spam, unactionable venting

### Priority Scoring Rules
- P0 (ship-breaking): affects >10% of users OR causes data loss OR security vulnerability
- P1 (next sprint): multiple users report the same issue OR blocks a common workflow
- P2 (backlog): single-user report OR nice-to-have improvement
- P3 (monitor): low signal, watch for recurrence

### Sentiment Scale
Score from -1.0 (extremely negative) to +1.0 (extremely positive).
Include a confidence score from 0.0 to 1.0.
Sarcasm, passive-aggressive tone, or "it would be great if [thing that is obviously broken]"
patterns should be scored as negative despite surface-level politeness.

### Known Issues (Do Not Re-Report)
- Plugin loading slow on Windows (PROJ-892, fix shipping in v2.3.1)
- OAuth token refresh race condition (PROJ-901, fix in progress)

This domain knowledge is what turns a generic LLM into a domain-aware classifier. The known issues list prevents the report from being dominated by already-tracked problems. The taxonomy ensures consistent categories across all five channels.

Step 3: Define the Worker Prompts

Each worker agent needs a channel-specific prompt. Add these as workflow sections in CLAUDE.md:

### GitHub Feedback Worker
When processing GitHub Issues feedback:
1. Fetch issues from repos [org/repo-a, org/repo-b] created or updated in the last 7 days.
2. Include: title, body, labels, reaction counts, comment count, author.
3. Exclude issues created by bots or with the "internal" label.
4. For each issue, output a JSON object:
   {
     "source": "github",
     "id": "org/repo#123",
     "title": "...",
     "summary": "one-line summary",
     "category": "from taxonomy",
     "sentiment": -0.6,
     "confidence": 0.85,
     "priority": "P1",
     "quote": "exact user quote that best represents the feedback",
     "signals": { "reactions": 12, "comments": 5, "duplicates": 2 }
   }

### Slack Feedback Worker
When processing Slack feedback:
1. Search channels: #support, #feedback, #general for messages in the last 7 days.
2. Filter to messages that contain product feedback (ignore social chat, meeting logistics).
3. Thread replies count as part of the parent message's feedback.
4. Same JSON output format as GitHub worker.

### Linear Feedback Worker
When processing Linear feedback:
1. Fetch issues with label "customer-reported" updated in the last 7 days.
2. Include: title, description, priority, status, customer quotes from comments.
3. Same JSON output format.

### Email Feedback Worker
When processing email exports:
1. Read feedback/email-export-*.csv from the last 7 days.
2. Parse columns: date, sender, subject, body.
3. Same JSON output format. Use sender email domain for signal weighting
   (company domains > personal emails for B2B products).

### App Store Review Worker
When processing app store reviews:
1. Read feedback/appstore-reviews-*.csv from the last 7 days.
2. Parse columns: date, rating, title, body, platform (iOS/Android).
3. Same JSON output format. Star rating maps to initial sentiment
   (1-2 stars = negative, 3 = neutral, 4-5 = positive), but override
   based on text content if they conflict.

The key design decision: every worker outputs the same JSON schema. This uniformity is what makes the synthesizer's job tractable. It does not care where the feedback came from -- it just sees a flat list of classified items.

Step 4: Define the Synthesizer Prompt

The synthesizer takes all worker outputs and produces the final report:

### Feedback Synthesizer
When I ask to synthesize feedback, after all workers have completed:

1. **Deduplicate.** Group items that describe the same underlying issue across channels.
   Use semantic similarity, not exact string matching. "CLI crashes on large files" from
   GitHub and "app freezes when I import big datasets" from Slack are the same issue.

2. **Cluster by theme.** Group related items into themes (e.g., "performance on large inputs",
   "plugin system reliability", "onboarding friction"). A theme needs 3+ items.

3. **Rank themes by urgency.** Score = (count of P0/P1 items * 3) + (total items * 1)
   + (sum of reaction/signal counts * 0.5). Higher score = higher urgency.

4. **Output the report** in this structure:

## Feedback Synthesis Report -- [date range]

**Sources processed:** [count per channel]
**Total items classified:** [number]
**Themes identified:** [number]

### Top Themes (ranked by urgency)

#### 1. [Theme Name] -- Score: [X] | Items: [N] | Channels: [list]
**Summary:** [2-3 sentences describing the pattern]
**Priority recommendation:** [P0/P1/P2]
**Evidence:**
- [source] [id]: "[quote]" (sentiment: X, confidence: X)
- [source] [id]: "[quote]" (sentiment: X, confidence: X)
- [source] [id]: "[quote]" (sentiment: X, confidence: X)
**Suggested action:** [concrete next step]

[repeat for each theme]

### Praise Highlights
- [source] [id]: "[positive quote]"
[top 5 positive items for team morale]

### Low-Signal Items (Monitor)
[items that appeared only once but might be early signals]

Step 5: Run the Pipeline

There are two ways to execute: sequential (simpler) and parallel (faster).

Sequential Execution

Run it as a single command:

claude -p "synthesize this week's feedback"

Claude Code reads the CLAUDE.md workflow, calls each worker sequentially through the MCP tool chain, then runs the synthesizer. This works but takes 2-3 minutes because each channel is processed one at a time.

Parallel Execution with Multiple Agents

For faster results, run each worker in a separate terminal pane:

# Pane 1: GitHub worker
claude -p "run the GitHub feedback worker, output to feedback/github-results.json"

# Pane 2: Slack worker
claude -p "run the Slack feedback worker, output to feedback/slack-results.json"

# Pane 3: Linear worker
claude -p "run the Linear feedback worker, output to feedback/linear-results.json"

# Pane 4: Email worker
claude -p "run the Email feedback worker, output to feedback/email-results.json"

# Pane 5: App Store worker
claude -p "run the App Store review worker, output to feedback/appstore-results.json"

Once all five complete, run the synthesizer:

claude -p "read all JSON files in feedback/*-results.json and run the feedback synthesizer"

The parallel approach cuts wall-clock time from 2-3 minutes to about 40 seconds. All five workers run simultaneously, and the synthesizer only processes the pre-classified output.

Step 6: Optimize for Cost

Running this pipeline weekly adds up. Here are three techniques that cut costs significantly.

Batch Processing

Use Claude's batch API for non-urgent weekly analysis. Batch requests run at 50% of standard pricing:

# Queue the analysis as a batch job (runs within 24 hours, 50% cheaper)
claude -p "synthesize this week's feedback" --batch

If you run synthesis every Monday morning, queue the batch job Sunday night. Results are ready by morning.

Prompt Caching

Your CLAUDE.md domain knowledge, taxonomy, and worker prompts are identical every week. Only the feedback data changes. Prompt caching means the static portion of the prompt (taxonomy, classification rules, output format) is cached after the first run. Subsequent runs only pay full price for the new feedback data.

For a typical weekly run with ~200 feedback items, prompt caching reduces input token costs by 60-70% because the CLAUDE.md instructions are the bulk of the prompt.

Token-Efficient Worker Output

The worker JSON schema is intentionally compact. Each worker produces structured summaries, not raw data dumps. The synthesizer processes maybe 2,000-3,000 tokens of worker output instead of 50,000+ tokens of raw feedback. This 10-20x reduction in synthesizer input is where the real savings happen.

Example Output

Here is what the pipeline produces for a fictional developer CLI tool:

## Feedback Synthesis Report -- 2026-03-15 to 2026-03-22

**Sources processed:** GitHub (34), Slack (28), Linear (15), Email (12), App Store (8)
**Total items classified:** 97
**Themes identified:** 7

### Top Themes (ranked by urgency)

#### 1. Plugin System Crashes on M-series Macs -- Score: 42 | Items: 11 | Channels: GitHub, Slack, Email
**Summary:** Users on Apple Silicon report that third-party plugins crash during
initialization. The crash is non-deterministic, appearing in roughly 30% of cold
starts. This started after the v2.3 release and correlates with the new plugin
sandbox introduced in that version.
**Priority recommendation:** P0
**Evidence:**
- github org/cli#487: "Plugin X crashes every other launch on my M2 MacBook,
  never happened on 2.2" (sentiment: -0.8, confidence: 0.92)
- slack #support: "Anyone else seeing plugins fail on Apple Silicon? Three
  people on my team hit this today" (sentiment: -0.7, confidence: 0.88)
- email support-4521: "We cannot roll out v2.3 to our team until this plugin
  crash is fixed. Blocking our adoption." (sentiment: -0.9, confidence: 0.95)
**Suggested action:** Investigate plugin sandbox memory allocation on ARM64.
Check if the v2.3 sandbox uses x86-assumed memory alignment.

#### 2. Config File Documentation Gaps -- Score: 18 | Items: 8 | Channels: GitHub, Slack, App Store
**Summary:** Users struggle to configure advanced features because the config
file reference is incomplete. The plugin system config options and environment
variable overrides are mentioned in the changelog but not in the docs.
**Priority recommendation:** P1
**Evidence:**
- github org/cli#501: "Where is the documentation for plugin config options?
  The changelog says 'see docs' but there is nothing there" (sentiment: -0.5,
  confidence: 0.90)
- appstore ios-2026-03-18: "Great tool but I spent 2 hours figuring out config.
  3 stars until docs improve." (sentiment: -0.3, confidence: 0.82)
**Suggested action:** Write config reference doc. Prioritize plugin system
and environment variable sections.

### Praise Highlights
- github org/cli#492: "The new pipe operator support is incredible. Cut my
  build script from 40 lines to 12."
- slack #general: "Just showed the team demo. Everyone wants to switch from
  our current tool."
- appstore ios-2026-03-20: "Best CLI tool I have used in 10 years. 5 stars."

### Low-Signal Items (Monitor)
- 1 email requesting Windows ARM support (no other signals yet)
- 1 Slack message about slow startup in Docker containers (could be environment-specific)

Tuning the Pipeline

After a few weekly runs, you will find areas to adjust.

Too many low-priority items. Raise the theme threshold from 3 items to 5. Add to your CLAUDE.md: A theme requires 5+ items to be included in the main report. Items below threshold go to the Low-Signal section.

Missing sarcasm. If the classifier marks polite-but-angry feedback as neutral, add more examples to the sentiment section of your CLAUDE.md. Real examples from your own feedback train better than generic instructions.

Duplicate themes. If "performance" and "speed" appear as separate themes, add a synonyms section: Treat these as the same theme: performance, speed, slow, latency, lag, hang, freeze.

Channel imbalance. If GitHub dominates because it has the most volume, add signal weighting: Weight items by signal strength: items with 5+ reactions or 3+ duplicates count as 2x. Single-occurrence items count as 0.5x.

The Multi-Pane Advantage

The parallel worker approach is where a multi-pane terminal layout proves its value. Five agents running simultaneously across five channels -- you want to see all of them working, catch errors early, and verify output quality before the synthesizer runs.

The ideal layout during execution: five small panes across the top, each showing one worker's progress. A larger pane at the bottom waiting for the synthesizer. When a worker finishes, you see its output immediately. If the Slack worker fails to authenticate, you catch it in the first 5 seconds instead of discovering it after the other four workers have finished.

After the pipeline stabilizes, the layout shifts to review mode: the synthesizer report in the main pane, raw worker outputs in side panes for spot-checking evidence quotes, and a terminal ready to act on the insights -- filing tickets, drafting responses, or updating your roadmap.

Extending the Pipeline

The same architecture adapts to adjacent workflows:

• Competitive analysis. Replace feedback channels with competitor review sites, Twitter mentions of competing products, and G2/Capterra reviews. Same clustering logic, different data sources.
• Release impact monitoring. After a major release, narrow the time window to 48 hours and add alert thresholds: if a P0 theme appears within 24 hours of release, trigger a Slack notification.
• Customer health scoring. Run per-customer instead of per-channel. Aggregate all feedback from a single customer across channels to build a health score over time.

Each extension reuses the same MCP connections and orchestrator-workers pattern. You change the CLAUDE.md prompts, not the infrastructure.

Recap

User feedback synthesis is a pattern recognition problem spread across too many tools. MCP servers connect Claude Code to all five channels. The orchestrator-workers pattern processes them in parallel. Domain knowledge in CLAUDE.md turns generic classification into product-aware analysis. Structured output with sentiment scores and evidence quotes makes the report actionable, not just informative.

The full setup: five MCP/data connections, one CLAUDE.md with domain knowledge and worker prompts, and one command to trigger the pipeline. Weekly feedback synthesis drops from a full PM workday to 60 seconds of compute and 10 minutes of review. The themes you catch in week 1 are the ones you would have missed until they became fires.