Groundline Evals

Four kinds of work, done by a dedicated team that learns your product

01 Build

Golden test sets

Realistic inputs for your agent, the correct answer for each, and a rubric for what counts as a pass.

02 Grade

Output review

We grade your agent's real outputs inside your own eval tool and tag every failure by type.

03 Run

Scenario testing

We run your agent through scripted tasks, from browser flows to spreadsheet work, and record where it breaks.

04 Calibrate

AI-grader checks

Two graders label the same cases independently, then we measure where your LLM judge disagrees with them.

Works inside Braintrust, LangSmith, Langfuse, Label Studio or a shared sheet. Your data stays in your tools.

Agents whose outputs have a right answer

We focus on agents where a careful, trained grader can check the work against a correct answer or a clear rubric.

Document and invoice extraction

Accounts-payable automation, AI bookkeeping, insurance claims intake, lending document review

Typical failures: drops line items on page 2, reads the subtotal as the total, swaps day and month on non-US dates.

We check: every extracted field against a hand-built correct answer.

Scheduling and email agents

AI assistants, recruiting coordination, field-service dispatch, sales follow-up tools

Typical failures: books the wrong time zone, misreads what the person asked for, sends a reply in the wrong tone.

We check: each action the agent took, against what the request actually needed.

Voice agents

Customer-support platforms, collections and mortgage servicing, appointment booking, sales calling

Typical failures: gives wrong policy details, talks over the caller, misses when to hand off to a person.

We check: recorded calls against your scorecard, and where your AI grader disagrees with us.

Compliance and risk review

Anti-money-laundering and KYC tools, fraud review, regulatory compliance software for banks and fintechs

Typical failures: closes a real alert as a false positive, or escalates a harmless one.

We check: each decision against an adjudicated answer, with the reason it was right or wrong.

Browser and computer-use agents

Web automation startups, portal and back-office automation, AI that operates internal tools

Typical failures: stuck on a login or pop-up, clicks the wrong item, reports success on an unfinished task.

We check: scripted tasks run in your sandbox, reviewed step by step from the recording.

Spreadsheet and finance copilots

Financial-modeling tools, FP&A software, AI analysts for finance and accounting teams

Typical failures: wrong cell references, hard-coded numbers, a forecast whose statements don't tie out.

We check: formulas and outputs against a correct version of the model.

What you receive at the end of a pilot

This is the format of a real deliverable. The agent, invoices and numbers below are invented for illustration.

Invoice extraction agent: evaluation report

50 invoices, double-graded, with AI-grader comparison

CLIENT: Example Co.
TEST SET: 50 invoices (20 digital, 15 scanned, 10 multi-page, 5 non-USD)
GRADERS: 2 independent + 1 adjudicator

39 / 50invoices fully correct (78%)

14errors found across 11 failing invoices

86%AI grader agreement with humans

94%agreement between the two human graders before adjudication

Accuracy by field

Field	Correct	Accuracy
Vendor name	49 / 50	98%
Invoice date	46 / 50	92%
Total due	45 / 50	90%
Line items (all present and correct)	36 / 50	72%

Where it fails

Failure type	Count	Where it shows up
missed_line_items	6	Multi-page invoices; items on page 2+ are dropped
total_is_subtotal	3	Takes the pre-tax subtotal as the total due
date_day_month_swap	2	Non-US invoices written DD/MM/YYYY
currency_dropped	2	EUR and GBP amounts returned with no currency
wrong_vendor_entity	1	Used the remit-to company instead of the seller

Graded cases

Can you trust your AI grader?

Your LLM judge agreed with human graders on 43 of 50 invoices (Cohen's kappa 0.61). It was reliable on vendor, date and total. It was not reliable on line items: it marked 5 of the 6 invoices with missing line items as complete, and failed 2 correct outputs.

Recommendations

Extract multi-page invoices page by page and merge, instead of in one pass. This targets 6 of 14 errors.
Add a deterministic check that line items plus tax equal the total. It would catch every total_is_subtotal and most missed_line_items errors.
Don't rely on the LLM judge for line items. Use the arithmetic check above, and keep the judge for vendor, date and total.
Add 10 more non-US invoices to the test set; date and currency errors cluster there.

Start with one two-week pilot

Day 0Mutual NDA. You pick one agent and give us access to its outputs in your eval tool.
Days 1–3We agree on a rubric with you and build a 50-case test set from your real traffic.
Days 4–10Two graders label every case independently; disagreements are adjudicated.
Days 11–14We compare your AI grader against the human labels and deliver the report and the graded set.

Pilot price $1,500

50 double-graded cases from your outputs
Failure breakdown and recommendations
AI-grader agreement report
The graded test set is yours to keep

Ask about a pilot

Who's behind this

Ethan Schneider, founder

Industrial and Systems Engineering at USC Viterbi, minor in AI applications, and an alum of the USC Marshall Greif Incubator. At a large tech company he calibrated an LLM grading rubric against about 4,600 labeled records. He runs every early project personally.

The team, in Nairobi

We're forming a dedicated team of IT graduates in Nairobi with a local AI solutions partner. The same people stay on your account, so they learn your product. Kenya is UTC+3, so graded work is often ready by the start of the US day.