A PE operating partner asked us to build production AI agents inside a portfolio company's billing system, processing real healthcare claims under HIPAA. Two people hand-wrote every rule in their claims engine across 300+ denial codes and payer logic that changes quarterly. Four months later, seven production agents handle it with zero patient data exposure. First month, we didn't touch a model. We mapped their data: where it sits and what's missing, so agents reason from structured facts instead of guessing. I've watched teams skip this step across dozens of engagements. They bolt a model onto the product, watch it hallucinate over unstructured inputs, and decide AI isn't ready for their industry. The data work is what makes it ready. We built an enrichment layer that assembles 34 dynamic variables per claim before any LLM sees it, pre-computed and versioned so the agent receives ranked facts instead of searching for context. Every agent follows one pattern: pre-compute context, strip all patient data before the model sees it, validate output against a strict schema, let deterministic code accept or reject the action. If the output falls outside the allowlist, the system fails closed. Seven agents, each locked to a single workflow like denied claim follow-up or billing reconciliation, each running its own enrichment payload. Then we built the eval harness. Every agent runs against a curated test suite before any update reaches production. When a model provider ships a new version or payer logic changes, the harness catches regression before a single live claim is affected. The flagship agent reconciles denials to the penny: 59 out of 60 on the eval set. Most teams launch an agent and hope it keeps working. We launch one and prove it does on every deployment. We route calls across two model providers. Swapping one changes nothing in the output because the eval harness verifies it. Model integration was the shortest line item in the four-month build. The operating partner now benchmarks the rest of the portfolio against this system. That's the line between a portfolio company running AI and one still running demos.
Dagens Vibes — 19. august 2026
Dagens feed handler mindre om endnu en model og mere om maskinrummet omkring den: strukturerede data, evals, færre round-trips og menneskelig dømmekraft. Agenterne er blevet nyttige nok til, at infrastrukturen pludselig betyder mere end demoen.
Fra X-feedet
Den stærkeste produktionshistorie: syv agenter behandler rigtige healthcare-claims under HIPAA. Første måned rørte holdet ikke en model; de byggede 34 versionerede variable pr. claim, fjernede patientdata, validerede mod stramme schemas og lod systemet fejle lukket. Modelintegrationen var den korteste linje i fire måneders arbejde. Sådan ser forskellen på en agent og en demo ud.
Hugging Face lod 1.221 mennesker og deres coding agents forsøge at reproducere 2.226 ICML-papers. 6.816 offentlige logbøger senere havde 23 procent mindst én falsificeret eller omstridt påstand — og de bedste resultater kom stadig fra mennesker, der styrede forsøget og stillede det irriterende spørgsmål én gang til.
Something super exciting happened quietly on HF over the past month: AI agents became AI builders, and they did it in the open! During our ICML reproduction challenge, 1,221 humans teamed up with coding agents to verify and reproduce 2,226 papers. But here's the cool part: everything happened on the @huggingface hub: 6,816 reproduction logbooks published openly, 2,962 cloud jobs launched, 35,908 claims judged, all traceable, all public and transparent For years the hub has been where humans collaborate on models, datasets and demos. Now we watch agents use it the same way: writing logbooks, publishing results, building on each other's work. Closed labs run evals behind closed doors and ask you to trust the press release. Open science means anyone can check the receipts and now agents can too! The next million users of the hub might not be human. and that might be the best thing to ever happen to science! Full write-up about the hackathon: https://t.co/8cgVzzEJnB
Addy Osmani rammer den nye engineering-rolle præcist: mennesker ejer produktintention, systemdesign og kvalitetsbar; tests, typer, mutation testing og arkitekturregler gør smagen maskinlæsbar. Flere checks er ikke kvalitet. Signal er kvalitet. Fabrikken skal stadig have en ansvarlig voksen.
If you're building a software factory, code good enough to ship still needs human taste and ownership. You'll likely need humans in the loop upfront for deciding on product intent, system design (if you care) and your quality bar. Do review code (lights-on factory) but be intentional with where it's needed the most. I've found you want to watch out for where automated back-pressure breaks. Or where maintainability trade-offs need to be made. Aim for quality checks to happen as early and continuously as possible. Not all of them have to, but this includes type systems, automated tests, mutation testing, security scanners and linting for architecture rules. Number of checks != quality. You'll likely need to experiment with what checks give you the best signal to noise ratio. Be ready to tighten or relax your constraints deliberately. You want to build your factory so some aspects of human taste get encoded in the environment, the agent gives you evidence of its work being right and where a human still "owns" what ships to production.
Dagens stille performance-gem havde tre likes og et bedre datapunkt end de fleste konferencetalks: at skære 12.000 tokens gjorde en agent billigere, men ikke hurtigere. At gå fra fire modelkald til ét sænkede svartiden fra 31,7 til 9,05 sekunder. Tæl round-trips før du påkalder prompt-voodoo.
I asked Agentwork "Who are the founders?" and it took 31.7 seconds to answer. Last Friday I sat down to figure out why. Quick background on how an agent like ours works: a model decides what to look up, calls a search tool, reads the result, decides the next step, and repeats until it can answer. My four-word question ended up as four model calls and four tool calls (two memory searches, one document fetch, one answer delivery). So the time can only be in a few places: the searches, the amount of text the model reads on every call, the number of back-and-forths with the model, or the model being slow to write. The first thing I went after was the reading. Every model call starts by taking in our full instructions plus descriptions of all 74 tools the agent can use. Around 104,000 tokens, a short novel, before your question even appears. Then I looked at the timings. All four searches together took 462 milliseconds. The other 29 seconds were the four model calls, one after another, each re-reading that novel to decide a single next step. Two things had made it this bad. The same approval-rules paragraph was pasted into all 74 tool descriptions, three times more text than the tool descriptions themselves. And we had never told the model how hard to think, so it was reasoning deeply over decisions like "search memory next". So I cut about 12,000 tokens per call. Wall clock: 31.7 became 31.8 seconds. Cheaper on every request, exactly zero seconds faster. The real lever was the back-and-forths. We now fetch the relevant memories before the first model call instead of letting the model discover its way there, and the double-check step only runs when the answer looks uncertain. Four model calls became one, and 31.7 seconds became 9.05. Of those, 6.68 is the single remaining model call. If your agent feels slow, count the model calls first. Next up is the 6.68 seconds - which is more complicated.
Vercel lægger op til én million dollar på bordet for folk, der kan slippe ud af deres Firecracker-sandbox eller bryde netværksgrænsen. Når agenter selv kan kortlægge gæstekernen, bygge en reproduktions-VM og skrive fuzzeren, er isolation ikke længere kedelig drift. Det er produktet.
Agents can now exploit vulnerable sandbox boundaries, so we are testing ours in the open. $1,000,000 hacker challenge for Vercel Sandbox: • Escape the Firecracker microVM • Defeat the host-side network boundary • Up to $50k/report via @Hacker0x01 https://t.co/6l4HL845jc
Den lille arbejdsform til sidst: en Claude-agent samler supporttickets, sorterer spam, triagerer features og vedligeholder en support.md plus kundernes knowledge hub. Jack skriver stadig svaret selv — bare ti gange hurtigere. Autonomi er overvurderet; overblik med turbo er bedre.
i can now do customer support from anywhere (phone or laptop) using claude app and i couldn't be happier about it. not just basic customer support wither, full on feature requests triaged, bug fixes + any other resolutions better than i ever could before 6 months ago this would have been a mess to think about doing but now its as simple as waking up one morning and telling your agent you need a better setup to do customer support from anywhere then giving it read only access to a few APIs and connect your customer support inbox to it. /support for me pulls all tickets and all memory of support hiccups goes into a support.md which is maintained and updated based on past tickets. which also connects to my knowledge hub that customers can see and updates it daily too - which leads to less support overall needed!!! most of my support inbox is actually spam and crap, auto sorted as such by my agent :D (guest posts, cold pitches, collab requests) I still add my own message back and handle everything myself so i know whats going on but it gets done 10x faster and more conveniently
Nyhedsbonus
Anthropic har fået wet-lab-resultater tilbage: Claude designede bindere mod 14 af 15 proteinmål, med hit rates på 22,6–35,1 procent mod typisk 10–15 procent. Eksterne laboratorier producerede og testede dem; samme udgivelse viser Opus 5 afkode rå NMR- og LC-MS-filer på under 25 minutter. Det er en sjældent håndgribelig AI-nyhed — molekylerne kan ikke presses til at like pressemeddelelsen.
OpenAI lancerer ChatGPT for Teens med automatisk teen-oplevelse ved estimeret alder under 18, Study Mode, homework-reminders, Study Hours og forældrekontroller. Den vigtige ændring er ikke endnu en policy-side, men at sikkerheds- og læringsvalgene bliver produkt-defaults.