# Use your history to choose the next growth test

One of my first jobs would be to understand which trial teams could benefit from a more useful invitation—and whether the opportunity is large enough to deserve Sales and Marketing Ops time.

This kit replays a specific targeting rule against historical account evidence. It separates three things: who would have been eligible, what actually happened later, and what we are assuming a new play might change. It does not estimate the play's causal effect.

## The proposed play

At trial day 3, offer the next useful step through the existing onboarding email program. The initial slice is accounts with an established eligible program-policy result, plausible company fit, and a first query against their own data, all available by that point. This is a relevant assistance audience, not a list of people who have already asked to speak to Sales.

Task and evaluation-intent signals shape the help and its priority; they are not universal entry requirements. An expressed team-evaluation need can justify a working-session offer. An explicit lack of evaluation intent favors self-serve help. Missing intent can call for a useful next-step prompt rather than an invented Sales opportunity. Company email eligibility comes from the company's established policy for this program; the tool neither requires an extra Sales opt-in nor infers eligibility from a missing permission field. Suppressed or unresolved accounts do not enter the contact audience.

A company match or product click alone does not establish intent. A meeting alone is not a qualified account. The proposed qualification definition requires Sales to confirm company fit, a real team use case, and an agreed next step; their team should adopt or revise it explicitly.

Marketing Ops can use the replay to inspect the eligible audience and exclusions. Sales can challenge the qualification criteria and available capacity before the test is built. The resulting work is a candidate play and a test brief, not an automated prospect campaign.

## What the working sample shows

The authored v2 synthetic sample contains 240 trial accounts arriving over 28 days. The SQL finds 160 eligible accounts and holds 80: 20 suppressed, 20 with unresolved email-program eligibility, 20 without established fit, and 20 without an own-data query available by day 3. Among the 160 with complete 28-day follow-up, 14 qualified historically: 8.75%.

Within the eligible audience, 100 accounts have stated evaluation intent, 20 have said they are not evaluating, and 40 have no usable intent signal at day 3. Task segments contain 120 recurring-analysis accounts, 20 with another task, and 20 with no known task. These are two overlapping descriptions of the same 160 accounts. Late intent remains unknown at the decision cutoff; it does not exclude the account from relevant help or retroactively change its day-3 segment.

Assuming a **five-percentage-point** improvement among all 160 eligible accounts implies eight additional qualified accounts: `160 × 0.05`. It changes a descriptive 8.75% rate to a modeled 13.75%. No measured lift supports that assumption. Five additional qualified accounts would require a 3.125-percentage-point improvement at this audience size.

At an assumed 15 minutes to review each candidate, the audience requires 40 gross review hours. Twelve available hours cover at most 48 candidates. That mismatch is a planning finding: narrow the review batch, change the process, or agree more capacity before building the play. It does not mean every recipient needs 15 minutes of Sales work; that is the editable review-process assumption being challenged. A 48-account batch is not evidence that an efficacy experiment would have enough statistical power.

Review hours exclude enrichment, outreach, preparation beyond the chosen review allowance, meetings, and follow-up. They are neither total Sales cost nor the incremental cost versus current practice. Adjust assumptions and include those other demands before making a resource commitment. Fractional modeled qualified accounts are arithmetic expectations, never a promise to deliver a fraction of a lead.

Program-policy results, segment signals and later outcomes are explicitly authored synthetic facts in this versioned sample. None describes MotherDuck's actual customer base, program rules or weekly volume. The qualification generator is independent of any proposed intervention; changing a targeting rule cannot establish that assistance caused a better outcome.

## Run your history with an agent

Start with the [two-server setup guide](agent-setup.md): Duck Yeah supplies the method and prepared SQL; your own MotherDuck connection supplies private schema access and query execution. The history replay uses plain SQL, with no model call or Jev fee. Normal MotherDuck compute usage, plan allowances and permissions apply.

Call `get_history_replay_template` with an explicit timezone-qualified `snapshotAt`, for example `{"snapshotAt":"2026-09-30T00:00:00Z"}`. Choose the analysis cutoff deliberately; that example is not a default. The public tool accepts only this date, never customer rows, schema, SQL or credentials.

The result includes `inputSchema`, the chosen cutoff, and four assembled SQL strings: `sql.mapping`, `sql.readiness`, `sql.candidates` and `sql.summary`. There are no nested template placeholders to resolve and no Python requirement. Privately replace the `YOUR_DATABASE.YOUR_SCHEMA.YOUR_REPLAYED_ACCOUNT_HISTORY` source with a mapped relation, or replace the `history_input` CTE with your reviewed mapping SELECT. Show the mapping before running it. Every query is a read-only SELECT; creating a warehouse table is unnecessary.

1. Inspect your schema and historical sources, and define one row per trial account.
2. Map the fields below using only information available at day 3; separately map the later outcome window.
3. Run `sql.readiness`. It reports missing, unknown, late and invalid values for all 17 fields after mapping. Counts overlap and are not a funnel. Outcome fields use the completed 28-day window and analysis cutoff, rather than incorrectly requiring outcomes at day 3. A missing `qualification_at` is expected for a confirmed non-qualification.
4. Resolve invalid mappings, inspect the candidate replay, then run the summary. Preserve unknown segments and incomplete outcomes. Historical facts that cannot be reconstructed produce a data-capture plan, not an invented rate.
5. Read task, intent and joint segment outcomes alongside their eligible, mature, qualified and unresolved counts. These three sets describe the same audience; do not add their totals together. Rates use mature accounts only and are null with no mature denominator. Missing follow-up and small cells limit interpretation.

Run the summary over the full mapped population. The candidate SELECT returns one row per trial account; inspect bounded samples with an outer `LIMIT`, paging or a private export. Do not place a limit inside the summary's replay CTE, and never calculate denominators from a tool response that may have been truncated.

Different historical rates can suggest where to investigate. They do not establish which segment will benefit from new assistance; similar rates do not refute personalized help either. That question needs a planned prospective comparison.

The prepared artifact is `history-replay-template.json`, generated from the canonical SQL by `node scripts/build-history-template.mjs`. Its status is `prepared_not_executed`. Local checks of the revised queries are separate from the original MotherDuck sample receipt. The saved `history-what-if.json` continues to describe its original input and query hash; that receipt does not verify these later query revisions.

## Optional: run the sample locally

Use a Python virtual environment with the project's pinned DuckDB dependency from `scripts/duck-yeah/requirements.txt`. The replay uses only DuckDB and Python's standard library; NumPy is needed for the separate experiment system, not this replay.

```sh
python scripts/duck-yeah/history_what_if.py
python scripts/duck-yeah/history_what_if.py \
  --uplift-pp 2 --review-minutes 6 --available-hours 12 --practical-extra 2
python -m unittest scripts/duck-yeah/test_history_what_if.py -v
```

The default command prints the summary and model without writing files. `--write-sample` recreates the deterministic public sample and artifact. A local regeneration records **local DuckDB** execution; it cannot retain or manufacture a MotherDuck verification claim.

The public sample is `public/duck-yeah/history-kit/sample.csv`. Its complete schema and generated values are inspectable. No customer or personal identifiers are needed; opaque account IDs suffice.

## Map your data inside MotherDuck

The kit is an input contract and SQL replay, **not a connector that knows an arbitrary warehouse schema**. Keep real records inside your own environment. There is no upload form or credential collection on this public page.

1. Start with `history_what_if_mapping.sql`. Map one row per trial account from your own warehouse's event/audit history. Use the historical state at day 3, including program suppression, policy changes, and changed fit or intent. Supply the company's dated eligibility decision for this onboarding program and its policy reference. Do not take today's account properties and pretend they were known earlier, or convert an old permission `no`/`unknown` into email eligibility.
2. Define each fact's `available_at` as the later of its occurrence and when your systems received it. An event occurring on day 2 but ingested on day 5 was not usable on day 3. If timing or evidence is unknown, preserve that uncertainty.
3. Map the separate 28-day outcome fields. A non-event requires complete observation; absence from a sparse event log is not a confirmed zero. Record the actual outcome definition and data gaps before interpreting a rate.
4. Either use the assembled MCP queries above, or generate the read-only readiness, candidate and summary queries with the optional Python emitter. It does not connect, create a view or execute SQL:

```sh
python scripts/duck-yeah/history_what_if.py \
  --source-relation analytics.growth.account_history \
  --as-of 2026-09-29T00:00:00Z --emit-sql readiness > replay-readiness.sql
python scripts/duck-yeah/history_what_if.py \
  --source-relation analytics.growth.account_history \
  --as-of 2026-09-29T00:00:00Z --emit-sql candidates > candidate-replay.sql
python scripts/duck-yeah/history_what_if.py \
  --source-relation analytics.growth.account_history \
  --as-of 2026-09-29T00:00:00Z --emit-sql summary > replay-summary.sql
```

Run those `SELECT` queries in your MotherDuck workspace. Replace the example analysis date deliberately. If there is no preexisting mapped relation, replace the `history_input` CTE with the mapping SELECT; no warehouse write is required. The template filenames are `history_what_if_replay.sql` and `history_what_if_summary.sql`.

5. Require `invalid_input_accounts = 0` before interpreting rates or choosing a play. Review the field diagnostics, held reasons and `unresolved_eligible_outcomes`. Then use the eligible count for workload and assumed improvement, and the mature outcome denominator for the historical rate. They are not interchangeable when follow-up is incomplete. Invalid outcome records remain in the eligible workload count and are marked unresolved; they cannot shrink the audience or become zeros. Invalid optional segment values become `unknown` and are flagged for correction without removing an otherwise eligible account or its valid outcome.

The original summary query, including its candidate-replay CTE, was executed as a read-only SELECT using synthetic values in the `duck_yeah_demo` MotherDuck context; the result matched local DuckDB. Verification metadata in `history-what-if.json` identifies that original query, not the revised readiness and segment-outcome queries in this kit. It does not imply a persistent MotherDuck table was created. All own-data queries still require real field mapping and validation.

### Input contract

All times are ISO instants with a timezone. Local CSV validation rejects timestamps without an offset. SQL mapping casts to `TIMESTAMPTZ`; configure your source mapping to UTC rather than allowing a server timezone to guess.

| Field | Meaning |
|---|---|
| `account_id`, `trial_started_at` | Unique account identifier and the start of its trial. Duplicate IDs invalidate the input. |
| `company_fit`, `company_fit_available_at` | `yes`, `no`, or `unknown`; when that historical value became usable. |
| `email_eligibility`, `email_eligibility_available_at`, `email_eligibility_policy` | Established company-program result: `eligible`, `suppressed`, or `unknown`; when it was available; and the policy result/reference. Eligibility needs a timely result with a nonblank reference. A present-day value or an unrelated Sales-consent field is insufficient. |
| `team_task`, `team_task_available_at` | `recurring_analysis`, `other`, or `unknown`, with historical availability. Describes the offer context; not a required positive signal. |
| `evaluation_intent`, `evaluation_intent_available_at` | Explicit `yes`, `no`, or `unknown`, with historical availability. Shapes the offer and priority; not a required positive signal. Do not infer it from enrichment. |
| `first_own_data_query_at`, `first_own_data_query_available_at` | Event time and availability time of the first successful own-data query. Both must be on or before day 3. |
| `qualification_outcome`, `qualification_at` | `qualified`, `not_qualified`, or `unknown`; the qualification event must be in `[trial start, trial start + 28 days)`. |
| `outcome_observed_through_at`, `outcome_recorded_at` | Evidence of complete follow-up and when the outcome was confirmed; both must support the 28-day window and the chosen analysis cutoff. |

Unknown categories remain `unknown`; missing timestamps remain empty/null. A required positive fact with no availability timestamp cannot make an account eligible. Missing or late task/intent signals become `unknown` segments and do not hold an otherwise eligible account. Eligibility uses no outcome column. Eligibility, outcome and segmentation validation issues are separate; the combined `validation_issue` still flags any invalid input. The SQL's held reasons are **mutually exclusive first failures**, in email-program result → fit → product-evidence order, after eligibility input/time validation. They reconcile to the held count; use readiness to see additional field gaps behind the first failure.

If your source has no historical audit trail for program eligibility or fit, this kit cannot reconstruct it. The result is a data-readiness finding and a capture plan. Missing historical task or intent limits segmentation, rather than proving the account should not receive relevant assistance. The timestamp checks cannot independently establish that a supplied timestamp, policy reference or historical value is truthful. Recheck current program eligibility and suppression before any future send; historical inclusion is not sending authority.

### Optional local analysis of your own CSV

If your organization permits an export, use the exact sample headers. The runner requires an explicit observation cutoff and a destination outside the site's public directory:

```sh
python scripts/duck-yeah/history_what_if.py \
  --input /private/work/account-history.csv \
  --output /private/work/history-plan.json \
  --as-of 2026-09-29T00:00:00Z \
  --uplift-pp 3 --review-minutes 10 --available-hours 20 --practical-extra 5
```

The JSON is labeled `owner_supplied_history`, contains aggregate results, and stays at your chosen path. If any eligible account has missing or immature outcomes, the qualification model is paused: `model.status` is `requires_complete_outcomes`, modeled extra accounts and rate are null, and only review workload remains. Complete or resolve follow-up rather than counting missing outcomes as zero or dropping them after seeing results. Complete cohorts use `model.status: ready`. The command neither uploads data nor contacts a provider. Its `defaults` field records the assumptions used for that output, and `model` carries their calculated consequences. Preserve your input hash and query versions with the result.

### Summary contract

`history-what-if.json` uses schema version 1. `history` contains total, eligible, held, and mature eligible counts; the observed qualified count/rate; unresolved outcomes; mutually exclusive exclusions; period; and descriptive eligible arrivals per week. `history.segments` adds `teamTaskAtDecision` and `evaluationIntentAtDecision` count maps for eligible accounts; each map sums to the same eligible count. `source` records the data kind, snapshot date, timing rule, and input hash. `verifiedExecution` names the engine actually used. `defaults` contains the assumed uplift in percentage points, gross review minutes, available hours, and desired additional qualified accounts. `model` supplies arithmetic, capacity, and the assumption needed to reach that goal. `nextExperiment` is a proposed test brief, not an approved or launched protocol. `kit` points to reusable source.

`eligiblePerWeek` is the count divided by the calendar-day span of these arrivals. It describes this sample period; it is not a forecast or evidence of steady arrival volume. For actual planning, inspect weekly cohorts, seasonality, acquisition mix, and the warehouse's explicit enrollment window.

The revised SQL additionally returns `invalid_outcome_accounts` and `segment_outcomes_json`. Each segment row includes `segment_family` (`task`, `intent`, `task_and_intent`), task/intent labels, `eligible_accounts`, `mature_accounts`, `qualified_accounts`, `unresolved_accounts` and `qualification_rate`. Each family reconciles independently to the same overall counts. Newly generated local JSON places these rows in `history.segments.outcomes`; the preserved earlier sample does not contain this newer field.

## The test this helps us prepare

Keep the current day-1 help. Among accounts eligible at day 3, randomly compare the current onboarding message with relevant assistance chosen by a fixed, documented rule from the available task and intent signals. That rule can offer self-serve help, a specific next-step prompt, or a team-evaluation conversation. Match contact opportunities across the arms. Decide the account/company identity boundary before assignment so linked users do not receive conflicting treatments. Analyze all assigned accounts, including those who do not respond.

The proposed primary outcome is the share becoming Sales-qualified by trial day 28. Define a worthwhile improvement, product-activation and continued-use protection, opt-out/complaint limit, review-time budget, enrollment cap, and follow-up period before launch. Check current program eligibility immediately before a send. Estimate the sample requirement and operating characteristics using their actual rate and volume; this small sample does not establish a production sample size. If available volume cannot resolve the business question, choose a feasible learning objective rather than lowering standards after seeing results. There is no universal 4,000-per-arm requirement.

The result should always generate another concrete decision:

- **Worthwhile improvement supported, protections pass:** Ops reproduces the play in one bounded expansion, retaining a comparison and quality/capacity checks. Next test whether a shorter preparation step preserves qualification while reducing review effort.
- **Worthwhile improvement ruled out, or unacceptable harm:** retire this version and review the leading rejection or friction reason. If teams want product help but see no value in a Sales conversation, test a self-serve team-use walkthrough against current help, using its own product-use outcome and protections. Other causes need a different revision.
- **Still inconclusive:** do not call it a loss or scale it. Check missing data and delivery, complete follow-up already owed, and continue only within the agreed enrollment and stopping plan. Otherwise close as unresolved and design the next study around what available volume can answer.

The branches are future decision plans. Selecting one does not alter the historical sample or create a measured result. Agents can help inspect SQL, interpret recorded rejection reasons, and draft the next test. They should cite the source evidence and leave missing facts unresolved; they cannot create intent, consent, qualification, or causal evidence.