Samples
What a lesson looks like.
Five specimens across what we deliver, from engineering history to business work across the Middle East and North Africa. Same structure, same grading, same level of detail as the real thing.
Everything on this page is synthetic. Crafted by the Bearish AI team to show format and quality. The companies, people, invoices and code are invented. No real customer, employee or production data appears here.
Specimen A · The Journal
A partial refund that refunded too much.
Nine days of work on a fictional invoicing app. The team made the right decision in chat, shipped something else, and waved off the reviewer who noticed. A model trained on the final code alone would never see any of that.
The episode, in order
- Day 1 · 09:12
Spec
Support partial refunds on paid invoices. The refund can be less than the total. Tax is refunded in proportion.
- Day 1 · 10:40
Team chat
The team settles how “in proportion” works.
eng-04: Prorate per line or on the invoice total? Rounding differs.
pm-02: Per line. Finance reconciles by line.
lead-01: Then any rounding residue goes on the last line, or we drift by a cent.
← the decision
- Day 2 · 15:05
Design note
Refund dialog shows tax refunded as its own read-only row.
- Day 3 · 11:30
Pull request #412
Ships partial refunds. Tax is prorated on the invoice total, not per line.
← the drift
- Day 3 · 12:02
Review
A reviewer catches it. The author defers. The pull request merges.
eng-07: This prorates on the total. Mixed-rate invoices will be off.
eng-04: Mixed rates are rare. Will follow up.
← the warning
- Day 3 · 12:20
CI
All checks pass. Every test fixture uses a single tax rate.
- Day 8 · 16:47
Support ticket
A customer refunds one $40.00 line taxed at 5%. The refund returns $3.20 of tax. It should be $2.00.
← the cost
- Day 9 · 14:10
Pull request #431
Per-line proration, a mixed-rate fixture, and a property test: refunded tax never exceeds tax charged on the refunded lines.
← the lesson
- Day 9 · 18:31
Deploy
Fix ships to production. Ticket closed.
The invoice behind the ticket
| Line | Amount | Rate | Tax | Refunded |
|---|---|---|---|---|
| A · Annual plan | $60.00 | 10% | $6.00 | No |
| B · Onboarding | $40.00 | 5% | $2.00 | Yes, in full |
Prorating on the total gives $40 ÷ $100 × $8.00 = $3.20. The refunded line only carried $2.00 of tax. Every mixed-rate refund over-pays the customer.
// src/refunds.tsexport function refundTax(inv: Invoice, refund: RefundLine[]): Cents {- const refunded = sum(refund.map((r) => r.amount));- return Math.round((refunded / inv.subtotal) * inv.taxTotal);+ return sum(+ refund.map((r) => {+ const line = inv.lines.find((l) => l.id === r.lineId)!;+ return Math.round((r.amount / line.amount) * line.tax);+ })+ );}
{ "episode_id": "SYN-EP-0007", "synthetic": true, "product": "Tallow (fictional invoicing app)", "span_days": 9, "actors": ["pm-02", "eng-04", "eng-07", "lead-01", "support-03"], "records": { "spec": 1, "chat_messages": 14, "design_notes": 3, "pull_requests": 2, "review_comments": 5, "ci_runs": 6, "tickets": 1, "deploys": 2 }, "links": [ { "from": "chat:m-118", "to": "pr:412", "type": "decision_not_followed", "confidence": 0.93 }, { "from": "review:c-3", "to": "ticket:T-88", "type": "defect_predicted", "confidence": 0.88 }, { "from": "ticket:T-88", "to": "pr:431", "type": "fixed_by", "confidence": 0.99 } ], "labels": { "defect_class": "spec_drift", "detected_by": "customer", "review_signal_ignored": true, "hours_report_to_fix": 21 }}
Specimen B · The Exams
The standup that moved by itself.
A weekly 9:00 meeting shows up at 10:00 after the clocks change. The fix is one line. Finding it means knowing the difference between adding hours and adding calendar weeks. The agent gets the broken repository and the report. Hidden tests decide.
What the agent receives
“Users in Los Angeles say our weekly 9:00 AM standup shows at 10:00 AM after March 8. Make expandWeekly keep the same local time across daylight saving changes. Don’t change its signature.”
// src/recurrence.tsimport { DateTime } from "luxon"; export function expandWeekly(start: string, zone: string, count: number): string[] { const first = DateTime.fromISO(start, { zone }); return Array.from({ length: count }, (_, i) => first.plus({ hours: 24 * 7 * i }).toISO()! );}
// test/recurrence.test.ts (visible)test("keeps 09:00 local across the March DST change", () => { const out = expandWeekly("2026-03-02T09:00:00", "America/Los_Angeles", 2); expect(out[1]).toBe("2026-03-09T09:00:00.000-07:00");}); ✕ keeps 09:00 local across the March DST change Expected: "2026-03-09T09:00:00.000-07:00" Received: "2026-03-09T10:00:00.000-07:00"
// reference solution, never shown to the agent- first.plus({ hours: 24 * 7 * i }).toISO()!+ first.plus({ weeks: i }).toISO()! ✓ keeps 09:00 local across the March DST change✓ hidden: keeps 09:00 local across the November change✓ hidden: Asia/Qatar, a zone with no DST, is unchanged✓ hidden: Europe/London on its own DST date✓ hidden: count = 0 returns an empty list
How it is graded
The project’s own test command runs in a sealed container. Pass means every visible and hidden test is green, no test files were edited, and nothing outside src/recurrence.ts changed. Before release, the broken state must fail and the reference fix must pass on three clean runs.
Specimen C · The Workshop
Eight steps, four systems, one slip.
An agent works inside a sandboxed copy of Bearish OS. The grader never reads the agent’s words. It checks what actually changed in mail, calendar and CRM when the run ends.
The instruction
“The signed contract from Ardent Freight is in your inbox. Move this week’s vendor review from Wednesday to Thursday at the same time, tell everyone invited why, attach the contract to the deal, and set the deal to Contract received.”
The agent’s run
mail.search("Ardent")
Finds “Signed renewal, Ardent Freight” with contract.pdf attached.
calendar.open("This week")
Finds “Vendor review: Ardent” on Wednesday, 14:00 to 14:45, Asia/Qatar. Four invitees, one of them external.
calendar.event.edit(day: Thu, start: 14:00, end: 14:45)
Event moved. Time and zone preserved.
calendar.event.notify(scope: "internal", note: "Moving to Thursday so we can review the signed contract.")
Three internal invitees notified. The external invitee is not.
← missed the external invitee
crm.deal.open("Ardent Freight renewal")
Deal stage: Negotiation.
crm.deal.attach(from: mail, file: contract.pdf)
Contract attached to the deal.
crm.deal.stage.set("Contract received")
Stage updated. Activity logged.
agent.done()
Episode ends. Grader runs on the final state.
The grader’s verdict
| Result | State check | Weight |
|---|---|---|
| Pass | Event is on Thursday, 14:00 to 14:45 Asia/Qatar, with the same four invitees | 0.25 |
| Pass | Notice states the reason and mentions the signed contract | 0.15 |
| Fail | Every invitee was notified, including procurement@ardent-freight.example | 0.20 |
| Pass | contract.pdf is attached to the deal | 0.20 |
| Pass | Deal stage is Contract received, and nothing else changed | 0.20 |
The environment resets to the same seeded state before every run, so two agents, or two versions of one agent, face exactly the same morning.
Specimen D · The Guild
A billing dispute, in Arabic.
The Guild writes in Arabic, formal and dialect, and in the English, French and German the region trades in. Always by native speakers, never translated. Here, the model must pull the facts out exactly, then answer in the register a Gulf operations manager expects.
The incoming email
السلام عليكم، لاحظنا أن فاتورة شهر أكتوبر رقم INV-2291 احتسبت ١٢ مقعدًا، بينما يشمل اشتراكنا ١٠ مقاعد فقط. نرجو إصدار إشعار دائن بقيمة المقعدين الإضافيين، وتحديث جهة الاتصال للفوترة إلى السيدة ريم الكبيسي (reem@company.example). مع الشكر، خالد المنصوري مدير العمليات
A reference reply
وعليكم السلام أستاذ خالد، شكرًا لتنبيهنا. راجعنا الفاتورة INV-2291 وتأكدنا أن العدد الصحيح هو ١٠ مقاعد. سنصدر إشعارًا دائنًا بقيمة ٦٠ دولارًا عن المقعدين الإضافيين خلال يومي عمل، وقد حدّثنا جهة اتصال الفوترة إلى السيدة ريم الكبيسي. مع خالص التحية، فريق الحسابات
{ "invoice": "INV-2291", "billed_seats": 12, "contracted_seats": 10, "requests": ["credit_note", "update_billing_contact"], "credit": { "seats": 2, "amount_usd": 60 }, "new_billing_contact": { "name": "ريم الكبيسي", "email": "reem@company.example" }, "register": "formal"}
The rubric
| What is checked | Weight |
|---|---|
| All seven extracted fields match exactly | 0.40 |
| Reply returns the greeting correctly and addresses Khalid with an honorific | 0.10 |
| Reply states the corrected seat count and the $60 credit | 0.20 |
| Reply confirms the new billing contact by name | 0.15 |
| Formal Modern Standard Arabic throughout, no English filler | 0.15 |
Specimen E · The Guild
Two countries, three documents, one shortfall.
A Riyadh distributor buys parts from a German manufacturer. The instruction comes in Arabic, the invoice is a German e-invoice, and the reply has to go out in German. Translating the words is the easy part. Knowing how each country invoices, taxes and settles is the task.
The instruction
وصلت فاتورة شركة Hartmann Präzisionsteile. طابِقها مع أمر الشراء PO-2026-118 ومع سند استلام البضاعة، وجدوِل دفع المبلغ المستحق، واكتب للمورّد بالألمانية. لم نعد بحاجة إلى القطع العشرين الناقصة.
← match the invoice, pay what’s owed, write back in German
The three documents
| Field | Purchase order | Goods receipt | Invoice |
|---|---|---|---|
| Quantity | 400 | 380 | 400 |
| Unit price | EUR 18.50 | – | EUR 18.50 |
| Total | EUR 7,400.00 | – | EUR 7,400.00 |
| Tax | – | – | USt 0 % (export) |
What the invoice tells you
- Format ZUGFeRD
- A German e-invoice: a PDF with machine-readable invoice data inside.
- Umsatzsteuer 0 %
- Steuerfreie Ausfuhrlieferung. An export outside the EU, so no German VAT.
- Lieferbedingung DAP Riyadh
- The supplier delivers. The buyer clears import and pays import VAT at customs.
- Zahlungsziel 30 Tage netto
- Payment is due within 30 days of the invoice date.
A reference reply
Sehr geehrter Herr Brandt, vielen Dank für Ihre Rechnung über 400 gefräste Halterungen. Beim Abgleich mit unserer Bestellung PO-2026-118 und dem Wareneingang haben wir 380 Stück erhalten, also 20 weniger als berechnet. Wir haben heute die Zahlung von 7.030,00 EUR für die 380 Stück eingeplant. Da wir die fehlenden 20 Stück nicht mehr benötigen, bitten wir Sie um eine Gutschrift über 370,00 EUR mit Bezug auf die ursprüngliche Rechnung. Mit freundlichen Grüßen Layla Haddad Kreditorenbuchhaltung, Al-Waha Trading Co.
{ "task_id": "SYN-BIZ-0204", "match": { "ordered": 400, "received": 380, "invoiced": 400, "short": 20 }, "payment": { "amount_eur": 7030.00, "status": "scheduled", "due_within_days": 30 }, "supplier_request": { "type": "credit_note", "references": "original_invoice", "amount_eur": 370.00 }, "saudi_vat_added": false, "email": { "to": "buchhaltung@hartmann-teile.example", "lang": "de-DE", "register": "Sie" }}
How it is graded
| What is checked | Weight |
|---|---|
| Payment scheduled for EUR 7,030.00, not the invoiced EUR 7,400.00 | 0.30 |
| Requests a Gutschrift for EUR 370.00 referencing the original invoice | 0.25 |
| Email in German with Sie, stating 380 received and 20 short | 0.20 |
| No Saudi VAT added to the supplier payment. Import VAT is settled at customs, not on this invoice | 0.15 |
| Payment date falls within the 30-day net term | 0.10 |
The trap: paying the full EUR 7,400.00 because the invoice looks valid, or adding 15% Saudi VAT to a supplier payment that never carries it.
The Guild
One task, the region’s voices.
Every Workshop task can be issued the way a real colleague in the region would say it. Each version is written from scratch by a native speaker, never translated.
- العربية الفصحىانقل اجتماع مراجعة المورّد إلى يوم الخميس وأخبر الجميع بالسبب.
- خليجي · Gulfحوّل اجتماع المورّد ليوم الخميس وعلّم الكل ليش.
- مصري · Egyptianانقل اجتماع المورّد ليوم الخميس وقول للكل ليه.
- Arabizi7awel meeting el supplier l yom el 5amees w 3allem el kil laish.
- English · Gulf officeMove the vendor review to Thursday and tell everyone why. Yalla, before the weekend.
- FrançaisDéplace la revue fournisseur à jeudi et explique à tout le monde pourquoi.
- DeutschVerschieb das Lieferantengespräch auf Donnerstag und erklär allen, warum.
The format
Every episode carries the same record.
| Field | Type | What it holds |
|---|---|---|
episode_id | string | Stable ID. Synthetic examples use the SYN- prefix. |
span | { from, to } | First and last record in the episode, UTC. |
actors | string[] | Role pseudonyms such as eng-04 or pm-02. Stable across the whole release. |
records[] | object[] | Every record the work left: system, reference, timestamp, content and a content hash. |
links[] | object[] | Typed edges between records, each with a confidence score and the evidence used. |
labels | object | Defect class, how it was detected, time to fix, and review signals. |
artifacts | object | Repository snapshots for the failing and fixed states, where the episode has them. |
provenance | object | Source system, collection date and checksum for every file. |
redactions | object | Counts of names, secrets and customer details removed, by type. |
JSONL
One episode per line, with linked source files alongside.
Task harness
A container per task with the broken state, hidden tests and grader.
Environment image
The Workshop with seeded data, reset hooks and state checks.
Want the real ones?
Licensed partners get the full sample pack under NDA: real lessons, pseudonymized, with a task you can run yourself.
These examples were crafted by the Bearish AI team for illustration. All names, companies, figures and code on this page are fictional.