get_before_after
Compares equal-length windows either side of a change date and returns an improved, regressed or no-detectable-change verdict.
When to use it
CRO is find, fix, then measure — this is the measure step. Run it after any theme deploy, price change, or fix shipped from a finding here.
Ask for it like this
You do not name the tool. These are questions that make an assistant reach for it:
Did my last theme deploy break anything?
Did the checkout fix I shipped on the 12th work?
How to read the result
- Store-scoped by default: session-based funnel counts plus a z-tested conversion comparison.
- Pass url to switch to page-scoped engagement metrics — bounce, scroll, duration, rage and dead clicks, p75 LCP — for that page.
- get_product_changes gives you candidate change dates, so you do not have to ask when something shipped.
What will make you wrong
Caveats
- This is not an A/B test. Traffic mix and campaigns move between windows.
- no_detectable_change means within noise at this volume, not "the change did nothing".
- Only the conversion z-test carries a significance claim. Count changes do not.
- If fewer than two full days have elapsed since the change date, the tool says so instead of comparing.
Parameters
changeDate
The date the change shipped, YYYY-MM-DD. The after window starts here.
daysBefore
Days before the change to compare. Both windows truncate to equal length.
daysAfter
Days after the change to compare.
url
Scope to one page, which switches the response to engagement metrics for that page.
Shared parameters it accepts
None. This tool is not date-scoped — it reports current state rather than a window.
Every tool also accepts merchantId, which resolves from your connection. You never pass it.
What comes back
JSON. These are the top-level keys; date-scoped tools additionally carry a _meta block with the server date and the data coverage envelope.
windows
The exact before and after date ranges compared.
conversion
Before and after rates, the z-test, and the verdict. Store-scoped only.
counts
Sessions, product-view sessions, add-to-cart sessions, orders and revenue for each window.
metrics
Page engagement metrics before and after. Page-scoped only.
note
Why this is not an A/B test, restated on every response.