compare-user-journeys

What this skill does

# Compare User Journeys

Investigate what distinguishes two user groups by pulling session replays and metrics for each, then producing a behavioral diff. This skill answers "what do winners do differently?" with concrete evidence from real sessions.

---

## CRITICAL: Tool Reference

**Primary tools:**
- **`Amplitude:get_session_replays`** — Find sessions for each user group using event count filters and user property filters. Run this twice — once per group.
- **`Amplitude:get_session_replay_events`** — Decode replays into interaction timelines. Run this for sessions from both groups.

**Supporting tools:**
- **`Amplitude:search`** — Search for existing charts, funnels, dashboards, events, and cohorts relevant to the comparison. Always check for existing analysis before building from scratch.
- **`Amplitude:get_cohorts`** — Discover existing cohorts that define the groups (e.g., "power users", "churned", "converted").
- **`Amplitude:get_events`** — Discover valid event names. Never guess event names.
- **`Amplitude:get_event_properties`** — Discover properties available for segmentation.
- **`Amplitude:query_chart`** / **`Amplitude:query_charts`** — Pull quantitative metrics segmented by the two groups for statistical grounding.
- **`Amplitude:get_feedback_insights`** / **`Amplitude:get_feedback_mentions`** — Check if feedback themes differ between groups.

---

## Instructions

### Step 1: Define the Two Groups

Parse the user's request to identify Group A and Group B. Common comparison patterns:

| Comparison | Group A | Group B |
|---|---|---|
| Conversion | Users who completed the goal | Users who didn't |
| Engagement | Power users / high-frequency | Casual / low-frequency |
| Retention | Retained users | Churned users |
| Plan tier | Enterprise / paid | Free / trial |
| Outcome | Successful (e.g., activated) | Failed (e.g., dropped off) |
| A/B test | Variant A | Variant B |

For each group, determine:
- **Defining criteria**: What makes a user belong to this group? (cohort, event, property)
- **Labels**: Clear names for the report (e.g., "Converters" vs. "Drop-offs", not "Group A" vs "Group B")

If the user's request is ambiguous (e.g., "compare power users"), ask: "What defines a power user for your product — is there an existing cohort, or should I use a frequency/engagement threshold?"

### Step 2: Get Context and Discover Segments

1. Call `Amplitude:get_context`. If multiple projects, ask which to compare within.
2. Call `Amplitude:get_cohorts` to check if existing cohorts match the requested groups. Existing cohorts encode institutional knowledge — prefer them over ad-hoc definitions.
3. Call `Amplitude:get_events` to discover events relevant to the comparison (the goal event for conversion comparisons, engagement events for activity comparisons, etc.).
4. Call `Amplitude:get_event_properties` for the key events to discover available segmentation properties.

### Step 3: Quantitative Comparison (Metrics)

Before watching replays, establish the statistical picture. Budget: 3-4 chart queries.

**First, search for existing analysis.** Call `Amplitude:search` with keywords related to the comparison (e.g., "onboarding funnel", "agent creation", "checkout conversion"). Existing charts and funnels encode institutional knowledge and save query budget. If you find a relevant funnel or segmented chart, use `Amplitude:query_chart` on it instead of building from scratch.

Use `Amplitude:query_chart` or `Amplitude:query_charts` to compare the two groups on:

1. **Volume & composition**: How many users are in each group? What's the ratio?
2. **Key metrics segmented by group**: Conversion rates, session frequency, feature adoption, retention — whatever metrics are most relevant to the comparison.
3. **Feature usage differences**: Which events does Group A fire significantly more than Group B? Use event totals or unique user counts segmented by group.

**What to look for:**
- Large percentage differences in specific event frequencies
- Features used exclusively or predominantly by one group
- Timing differences (Group A does X on day 1, Group B waits until day 7)
- Volume differences (Group A does X 5 times per session, Group B does it once)

Record the top 3-5 quantitative differences — these become hypotheses to validate or enrich with replays.

### Step 4: Find Sessions for Both Groups

Run `Amplitude:get_session_replays` **twice** — once for each group. Request `limit: 8` for each.

**Exclude internal users by default.** Unless you're specifically studying internal behavior, add a user property filter to exclude your company's email domain (e.g., `gp:email does not contain "amplitude.com"`). This prevents internal test sessions from polluting the comparison.

**Building group filters:**

If groups are defined by a **cohort**, filter on the cohort's defining user property or event.

If groups are defined by an **event outcome** (e.g., completed checkout vs. didn't):
- **Group A (converters)**: Filter for sessions containing the conversion event (use `eventCountFilters` with `operator: "greater or equal"`, `count: "1"`).
- **Group B (drop-offs)**: Filter for sessions containing the entry event but NOT the conversion event. Since `get_session_replays` doesn't support "did not do" filters directly, filter for the entry event only, then when watching replays in Step 5, check the interaction timeline for absence of the conversion event. Discard sessions where the user actually converted and replace with additional sessions until you have 4-6 confirmed drop-offs.

If groups are defined by a **user property** (plan, segment):
- Use user property filters with `event_type: "_all"`.

### Step 5: Watch Sessions — Extract Interaction Timelines

For each group, call `Amplitude:get_session_replay_events` for 4-6 sessions. Use `event_limit: 300`.

**Budget: 8-12 total sessions (4-6 per group).**

**Rate limiting and large responses:**
- Call `get_session_replay_events` in batches of 3-4 at a time, not all at once. The Session Replay API enforces concurrency limits and will return 429 errors if overwhelmed. If you hit a 429, wait briefly and retry.
- Some sessions have extremely high interaction counts (100K+ raw events). If a response is too large and gets saved to a file, read the key portions from the saved file path. For sessions with very high `total_raw_events`, consider using a lower `event_limit` (e.g., 150) to stay within response size limits.

**While analyzing, track these behavioral dimensions for each session:**

| Dimension | What to capture |
|---|---|
| **Navigation path** | Sequence of pages visited. Note the order and any pages unique to this session. |
| **Feature engagement** | Which features/areas the user interacted with. Note depth (quick glance vs. extended use). |
| **Pace & hesitation** | Fast and confident vs. slow with pauses. Note long gaps between actions. |
| **Exploration vs. focus** | Did the user go straight to their goal or browse around? |
| **Friction encountered** | Errors, rage clicks, back-navigation, abandoned inputs. |
| **Session outcome** | Did they accomplish something? What was the last action before leaving? |

For each session, write a 2-3 sentence behavioral summary capturing the overall pattern.

### Step 6: Synthesize the Behavioral Diff

This is the core analytical step. Compare the two groups across the dimensions above.

1. **Tabulate patterns.** For each behavioral dimension, summarize what Group A typically does vs. Group B.

2. **Identify differentiating actions.** Find behaviors that are:
   - **Present in Group A, absent in Group B** — These are the strongest signals. "Converters explore the template gallery before creating their first project. Drop-offs skip it entirely."
   - **Present in both but with different frequency or depth** — "Both groups use search, but power users average 4 searches per session vs. 1 for casual users."
   - **Present in Group B, absent in Group A** — Negative signals. "Chur