How we measured: ProLyt vs. pasting your tables into a chatbot
Test run 2 October 2026 (Opus pasting run 3 October) · Models: Claude Sonnet 5.5 and Claude Opus 5.5 (Anthropic API) · Prices from Anthropic's public price list, fetched 2 October 2026.
The question we asked
When you want answers from your company's tables, there are two common ways to use an AI model:
- Paste the data in. Export the table to CSV, paste it into the prompt, ask your question. The model reads every row, every time.
- Let ProLyt do it. ProLyt gives the model your table structure (names, columns, types), the model writes SQL, your database computes the numbers, and the model explains the result. Your full tables are never put into the prompt; only query results (a limited number of rows) are.
We measured how many tokens each approach uses for the same question, and whether the answer was right.
Setup
- Data: a real multi-table dataset of 13 tables (46,213 rows and 468 columns in total).
- Questions, from simple to complex:
- Q1: one total over one table, filtered to one entity and one period.
- Q2: a join of two tables with grouping and a ranking.
- Q3: a question across two tables with a conditional comparison, split two ways.
- Data each question needs:
| Question | Tables needed | Rows pasted | Columns pasted |
|---|---|---|---|
| Q1 | 1 | 682 | 59 |
| Q2 | 2 | 1,364 | 79 |
| Q3 | 2 | 1,023 | 88 |
- Correct answers were computed once with hand-written SQL. Every run was graded right or wrong against them, then all grades were checked by hand.
- Paste-in approach: each prompt contained exactly the tables that question needs, plus the question. That is the most favourable case for pasting: in real life you often don't know which tables you need and paste more.
- ProLyt: the same questions, with all 13 tables available. ProLyt had to find the right tables itself.
- Measurement: token counts are the ones reported by Anthropic's API for each call, captured by a pass-through logger. Nothing was estimated.
Results (Claude Sonnet 5.5)
| Question | Approach | Correct | Input tokens per question | Cost per question |
|---|---|---|---|---|
| Q1 | Paste the table in | 1 of 1 | 192,600 | $0.400 |
| Q1 | ProLyt | 2 of 2 | 19,800 | $0.043 |
| Q2 | Paste the tables in | 0 of 1 (no answer, see notes) | 270,600 | $0.701 |
| Q2 | ProLyt | 2 of 2 | 55,000 | $0.126 |
| Q3 | Paste the tables in | 0 of 1 (no answer, see notes) | 232,100 | $0.624 |
| Q3 | ProLyt | 2 of 2 | 78,300 | $0.186 |
Prices used (Claude Sonnet 5.5, per million tokens): input $2.00, output $10.00. ProLyt figures are the mean of its 2 runs.
What it shows
- Q1, the like-for-like comparison (both approaches answered correctly): ProLyt used 9.7× fewer input tokens and cost about 9× less per question.
- Across all three questions, ProLyt used 3× to 10× fewer input tokens. More complex questions need more steps (more model calls), so the gap narrows on them.
- ProLyt answered all 6 runs correctly. Its numbers come from SQL executed on the data, not from the model reading rows.
What it means in money
Token counts become dollars at the model's list price. For Q1, the question both approaches answered correctly:
| Model | Pasting the table in | ProLyt | Saved per question | Saved per 1,000 questions |
|---|---|---|---|---|
| Claude Sonnet 5.5 ($2 / $10 per million tokens) | $0.400 | $0.043 | $0.36 | ≈ $360 |
| Claude Opus 5.5 ($4 / $20 per million tokens) | $0.812 | $0.088 | $0.72 | ≈ $720 |
All figures in this table are measured; each approach answered Q1 correctly on both models. The Opus figures are one run each.
- The ratio stays about 9× on both models. The dollars saved grow with the model's price and with the size of your tables, because pasting pays for every row on every question.
- Per-1,000 figures assume questions like Q1. More complex questions showed 3–5× fewer tokens with ProLyt, so smaller but still substantial savings per question.
- These savings apply to an AI API bill — for example when you use your own API key, or compare against building the paste-in workflow yourself. On a ProLyt plan, model usage is included in a flat monthly price: you never pay per token or per row.
Why the gap grows with your data
Pasting scales with table size: twice the rows, roughly twice the tokens, on every question, until the data no longer fits in the model's context at all. ProLyt's prompt scales with your question and schema, not with row count. The tables in this test are small (under 1,400 rows per question); on typical company tables with tens of thousands of rows, pasting costs proportionally more, while ProLyt's cost per question stays roughly flat.
Honest limits of this test
- Small sample. 3 questions, 1–2 runs each. Treat the numbers as indicative, not statistical.
- Q2 and Q3 paste-in runs produced no answer because the model used its whole response budget (16,000 tokens) reasoning before writing. With a larger budget it may well have answered. So we make no claim that pasting gets these questions wrong, and we compare cost only on Q1.
- The model in these runs was Claude. ProLyt was connected to Claude Sonnet 5.5 (and Opus 5.5 for one Q1 run) for this test. ProLyt's plans run on open models; the correctness figures above are Claude's and are not a claim about those models.
- ProLyt's cost here is an upper bound. The version tested did not yet use prompt caching for Anthropic models. With caching, repeated parts of the prompt are billed at a tenth of the input price or less.
- One dataset, one date. Prices and models change; we will re-run and update this page.
- Self-run test. We built ProLyt and ran this test ourselves. The method is described here so you can reproduce it.
Reproduce it yourself
- Pick a question and the table(s) it needs. Export them as CSV.
- Paste them into the model with the question; note the input tokens the API reports.
- Ask ProLyt the same question on the same data; ProLyt reports the SQL it ran and the tables used.
- Check both answers against a number you computed yourself.