general-analysis

What this skill does

# Fullstory Analytics

## Mental Model

Internalize these three concepts before choosing tools:

- **Segment** = a cohort of users (the "who"). A segment is a filter, not a measurement. It narrows which users' data a metric runs against.
- **Metric** = the measurement (the "what" and "how much"). Every quantitative answer is a metric. Even "how many users visited /checkout" is a metric (count of page views), optionally filtered by a segment.
- **Session** = evidence (the "why"). Sessions are qualitative. Use them to understand *why* a number looks the way it does — not to answer the quantitative question itself.

## Step 0: Classify Intent

Before calling any tool, determine what the user is asking for:

- "how many", "what's the count", "what percentage", "what's the rate" → quantitative answer → `single_number` metric
- "which pages", "top N", "by browser", "breakdown by" → breakdown → `top_n` metric
- "over time", "by day", "is it getting worse", "trend" → trend → `trend` metric
- "mobile vs desktop", "compare", "A vs B" → comparison → invoke the `comparisons` skill
- "show me sessions", "let me watch", "examples of" → session exploration → `fullstory:get_sessions` with `metric_id`
- "sessions from power users", "show me what enterprise users do" → cohort browsing → `fullstory:build_segment` then `fullstory:get_sessions` with `segment_id`

If the intent is ambiguous, ask the user before proceeding. Getting the intent wrong wastes a build+compute cycle.

## Step 1: Resolve or Build

### Always search before building

Users often don't know what metrics or segments already exist in their Fullstory account. Always search first, even when the question sounds ad-hoc. Use `fullstory:get_metric(regex="...")` or `fullstory:get_segment(regex="...")`, starting broad and narrowing if needed (e.g., "how many rage clicks on checkout?" → start with `checkout`, then try `checkout.*rage` if the first search returns too many results).

Results include a short description of the segment's filters and events, so use that — not just the name — to judge relevance. If no results match, tell the user nothing was found and confirm before building. If results come back but their filters/events don't match the question, tell the user what you found and that none seem to match, then confirm they'd like you to build a new one.

If 2 or more plausible candidates come back, immediately call `fullstory:get_view_count` on their IDs (up to 10) to rank by popularity. If search returns more than 10 candidates, pass the 10 most name-similar IDs. Then:

- If one candidate has clearly more views (roughly 5x or more than the next), treat it as the canonical object — proceed with it and tell the user you're using "the most-used version."
- If the top 2–3 are comparable in view count, present them sorted by popularity. Use the filters, events, and description fields from the search results to explain what each one measures or captures differently, then ask the user which to use.
- If all candidates have zero or near-zero views, flag them as likely stale and offer to build fresh.

### Building new

**Metrics:** Before building, make sure the unit of measurement is correct — getting this wrong is the most common source of misleading results. If the question is about "customers", "accounts", or "organizations", clarify whether the user wants to count individual users or group users by a customer/account/organization property. If it's the latter, look for user properties that match and build the metric to count by that property. Similarly, watch for ambiguity between pages and URLs — "which pages" usually means page titles or paths, not full URLs with query parameters.

Call `fullstory:build_metric` with a descriptive query and the correct `output_type` derived from intent classification:

- Quantitative answer → `single_number`
- Breakdown → `top_n`
- Trend → `trend`

For `top_n`, make sure the grouping dimension is expressed in the query (e.g., "top pages by rage click count"). The metric builder will not invent a dimension on its own.

**Segments:** Call `fullstory:build_segment`. Always reference by `segment_id` in subsequent steps. If the same cohort is needed for multiple questions in the conversation, reuse the existing `segment_id` — do not rebuild.

### Refining existing

If the user wants to modify a metric or segment already established in this conversation — adding or removing a filter, changing aggregation, adjusting the time range, or changing output shape — use `fullstory:update_metric` or `fullstory:update_segment`. Pass the existing `metric_id` or `segment_definition` and a natural language `refinement`.

- `fullstory:update_metric`: accepts `metric_id` and supports two mutually exclusive modes: LLM refinement (filter changes, aggregation changes, output type overrides via `output_type`) and segment attachment (attach a `segment_id` to the metric). Does not support ratio metrics — rebuild those with `fullstory:build_metric`.
- `fullstory:update_segment`: supports filter additions/removals and time range changes.

## Step 2: Compute

Call `fullstory:compute_metric` with:
- `metric_id` — the ID returned by `fullstory:build_metric`, `fullstory:get_metric`, or `fullstory:update_metric`
- `time_range` — default is `last_30_days`; ask the user if they want a different window

If the question is scoped to a cohort, segments must be pre-attached before computing. Call `fullstory:update_metric(metric_id, segment_id)` first, then call `fullstory:compute_metric(metric_id)`. Do not pass `segment_id` directly to `fullstory:compute_metric`.

Present results in plain language with context:
- Numbers: "12,340 dead clicks over the last 30 days"
- Tables: highlight the top entries; include percentages if a total is available
- Trends: call out direction, magnitude, and any inflection points

Always surface `metric_url` so the user can verify in the Fullstory UI.

## References

Load these when the situation calls for it:

- `references/validation.md` — when results are zero, anomalous, or the user expresses skepticism
- `references/sessions.md` — when investigating sessions to understand why a metric looks the way it does

## Guidelines

- Default `time_range` is `last_30_days`. Ask before using a different window unless the user specified one.
- When building a segment for use in a later step, always reference by `segment_id`.
- Reuse `segment_id` and `metric_id` within a conversation. Do not rebuild objects the user has already established.
- If the user asks for a different shape of an existing metric (e.g., they have a count but now want a trend), call `fullstory:update_metric` with the existing `metric_id` and the desired `output_type`. Only fall back to `fullstory:build_metric` for fundamentally different queries or ratio metrics.
- When presenting table results, include both the dimension value and the count. If a total is available, show percentages.
- Always surface `metric_url` in your response so the user can verify in the Fullstory UI. `fullstory:build_metric` and `fullstory:update_metric` both return `metric_url` — surface it as soon as it's available, don't wait until after computing.