Dashboard guide
Success metrics
Define what counts as success as a single number.
Success metrics is the stop you make before an experiment starts. ABTO measures cost and speed on its own, but success differs by product and use case, so you decide here what counts as success. Without that bar, Compare has no way to tell you which variant did better.
The metric list and the event table
Section titled “The metric list and the event table”| Area | What you see |
|---|---|
| Metric list | The metrics defined here, one of them marked Primary |
| Metric editor | Choose the form, ratio or value, then pick the events or write the formula. Edits save automatically |
| Event table at the bottom | Events sent by the client SDK appear automatically. Each shows its occurrence rate for the period and the metrics that use it |
If an event you want is missing from that table, your site or app is not sending it yet.
One line, abto.capture('summary_copied'), and it appears from the next occurrence.
The two metric forms
Section titled “The two metric forms”| Form | What it measures | Example |
|---|---|---|
| Ratio | The share of people who received an AI call and did the target behavior. Pick up to five events and weight them | Summary success rate = purchase 70% + summary_copied 30% |
| Value | One number tying together call-side figures and numbers you send on events through a formula | Revenue per AI dollar = sum of purchase value ÷ sum of call cost |
A ratio counts people, not calls, so it always stays between 0 and 100%. The call-side figures available to a value are cost, tokens, latency, and request count, aggregated as sums, averages, or percentiles alongside the numbers you send on events. Request count and latency follow the same rules as request count and average latency in Compare. A value metric like “revenue per AI dollar” answers “is the AI paying for itself” directly.
The Primary metric
Section titled “The Primary metric”Mark one metric as Primary. The first metric becomes Primary automatically. The Primary metric feeds the Overview summary numbers and chart, and becomes the default basis on Compare. To prevent deleting it by accident, the Primary metric cannot be deleted.
Better when
Section titled “Better when”A setting on ratio metrics and on the value metrics the Results tab judges, below the metric definition. The Results tab in Compare reads verdicts in this direction. A metric whose first operand is call cost, tokens, or latency starts as Lower is better, and every other metric starts as Higher is better. Switching the first operand between call cost, tokens, or latency and anything else changes the direction to match. The Results tab section of the Compare page lists which value metrics are judged.
- Higher is better: Rising above the baseline variant is an improvement.
- Lower is better: Falling below the baseline variant is an improvement. This fits metrics that should go down, such as a repeat-inquiry rate.
What makes a good metric
Section titled “What makes a good metric”Define success not by whether the response merely looked good, but by what users did after receiving it: saving, purchasing, sharing. Including one behavior close to the feature speeds up the verdict: for review summaries, copying the summary reacts to changes faster than a final purchase. Start simple with a one-event ratio metric, and evolve it into a weighted blend as you operate.
With one metric in place, you are ready to screen candidates in Testing.