# How we measured: ProLyt vs. pasting your tables into a chatbot

_Test run 2 October 2026 (Opus pasting run 3 October) · Models: Claude Sonnet 5.5 and Claude Opus
5.5 (Anthropic API) · Prices from Anthropic's public price list, fetched 2 October 2026._

## The question we asked

When you want answers from your company's tables, there are two common ways to use an AI model:

1. **Paste the data in.** Export the table to CSV, paste it into the prompt, ask your question. The
   model reads every row, every time.
2. **Let ProLyt do it.** ProLyt gives the model your **table structure** (names, columns, types), the
   model writes SQL, your **database computes the numbers**, and the model explains the result. Your
   full tables are never put into the prompt; only query results (a limited number of rows) are.

We measured how many tokens each approach uses for the same question, and whether the answer was
right.

## Setup

- **Data:** a real multi-table dataset of **13 tables** (46,213 rows and 468 columns in
  total).
- **Questions**, from simple to complex:
  - **Q1:** one total over one table, filtered to one entity and one period.
  - **Q2:** a join of two tables with grouping and a ranking.
  - **Q3:** a question across two tables with a conditional comparison, split two ways.
- **Data each question needs:**

| Question | Tables needed | Rows pasted | Columns pasted |
|---|---|---|---|
| Q1 | 1 | 682 | 59 |
| Q2 | 2 | 1,364 | 79 |
| Q3 | 2 | 1,023 | 88 |

- **Correct answers** were computed once with hand-written SQL. Every run was graded right or wrong
  against them, then all grades were checked by hand.
- **Paste-in approach:** each prompt contained **exactly the tables that question needs**, plus the
  question. That is the most favourable case for pasting: in real life you often don't know which
  tables you need and paste more.
- **ProLyt:** the same questions, with **all 13 tables** available. ProLyt had to find the right tables
  itself.
- **Measurement:** token counts are the ones **reported by Anthropic's API** for each call, captured
  by a pass-through logger. Nothing was estimated.

## Results (Claude Sonnet 5.5)

| Question | Approach | Correct | Input tokens per question | Cost per question |
|---|---|---|---|---|
| Q1 | Paste the table in | 1 of 1 | 192,600 | $0.400 |
| Q1 | **ProLyt** | **2 of 2** | **19,800** | **$0.043** |
| Q2 | Paste the tables in | 0 of 1 (no answer, see notes) | 270,600 | $0.701 |
| Q2 | **ProLyt** | **2 of 2** | **55,000** | **$0.126** |
| Q3 | Paste the tables in | 0 of 1 (no answer, see notes) | 232,100 | $0.624 |
| Q3 | **ProLyt** | **2 of 2** | **78,300** | **$0.186** |

Prices used (Claude Sonnet 5.5, per million tokens): input $2.00, output $10.00. ProLyt figures are
the mean of its 2 runs.

### What it shows

- **Q1, the like-for-like comparison** (both approaches answered correctly): ProLyt used **9.7× fewer
  input tokens** and cost **about 9× less** per question.
- **Across all three questions**, ProLyt used **3× to 10× fewer input tokens.** More complex questions
  need more steps (more model calls), so the gap narrows on them.
- **ProLyt answered all 6 runs correctly.** Its numbers come from SQL executed on the data, not from
  the model reading rows.

## What it means in money

Token counts become dollars at the model's list price. For **Q1**, the question both approaches
answered correctly:

| Model | Pasting the table in | ProLyt | **Saved per question** | **Saved per 1,000 questions** |
|---|---|---|---|---|
| Claude Sonnet 5.5 ($2 / $10 per million tokens) | $0.400 | $0.043 | **$0.36** | **≈ $360** |
| Claude Opus 5.5 ($4 / $20 per million tokens) | $0.812 | $0.088 | **$0.72** | **≈ $720** |

All figures in this table are measured; each approach answered Q1 correctly on both models. The Opus
figures are one run each.

- **The ratio stays about 9× on both models.** The **dollars saved grow with the model's price** and
  with the **size of your tables**, because pasting pays for every row on every question.
- Per-1,000 figures assume questions like Q1. More complex questions showed 3–5× fewer tokens with
  ProLyt, so smaller but still substantial savings per question.
- These savings apply to an **AI API bill** — for example when you use your own API key, or compare
  against building the paste-in workflow yourself. **On a ProLyt plan, model usage is included in a
  flat monthly price: you never pay per token or per row.**

## Why the gap grows with your data

Pasting scales with **table size**: twice the rows, roughly twice the tokens, on **every** question,
until the data no longer fits in the model's context at all. ProLyt's prompt scales with your
**question and schema**, not with row count. The tables in this test are small (under 1,400 rows per
question); on typical company tables with tens of thousands of rows, pasting costs proportionally
more, while ProLyt's cost per question stays roughly flat.

## Honest limits of this test

- **Small sample.** 3 questions, 1–2 runs each. Treat the numbers as indicative, not statistical.
- **Q2 and Q3 paste-in runs produced no answer** because the model used its whole response budget
  (16,000 tokens) reasoning before writing. With a larger budget it may well have answered. So we
  make **no claim** that pasting gets these questions wrong, and we compare cost only on Q1.
- **The model in these runs was Claude.** ProLyt was connected to Claude Sonnet 5.5 (and Opus 5.5
  for one Q1 run) for this test. ProLyt's plans run on open models; the correctness figures above
  are Claude's and are not a claim about those models.
- **ProLyt's cost here is an upper bound.** The version tested did not yet use prompt caching for
  Anthropic models. With caching, repeated parts of the prompt are billed at a tenth of the input price or less.
- **One dataset, one date.** Prices and models change; we will re-run and update this page.
- **Self-run test.** We built ProLyt and ran this test ourselves. The method is described here so you
  can reproduce it.

## Reproduce it yourself

1. Pick a question and the table(s) it needs. Export them as CSV.
2. Paste them into the model with the question; note the input tokens the API reports.
3. Ask ProLyt the same question on the same data; ProLyt reports the SQL it ran and the tables used.
4. Check both answers against a number you computed yourself.
