- TypeSafe Jev
- System One
- email agents
- cost architecture
Email Agents: Put Organize/Sort/Score on a Decision Layer, Not a Chat LLM
I moved organize, sort, and score in my email agent onto TypeSafe Jev ($0.042/M input, output free). The cost win is splitting decision work from generation, not finding a cheaper chat model.
I was calling a frontier LLM for every organize, sort, and score step in my email agent.
That looked reasonable when the agent was a prototype. One model read the message, labeled it, ranked it, and sometimes drafted a reply. The bill and the latency told a different story: most of those calls never produced a sentence a human would read. They produced a folder, a priority, a route.
TypeSafe AI's System One model Jev (jev-1.13.0) is built for that second job. Their published list price is $0.042 per million input tokens with output free. You POST state plus typed questions to api.typesafe.ai/v1/systemone and get choice, score, or noul answers with probabilities. No chat reply. No coding agent. Typed decisions for software.
I moved the organize/sort/score path onto Jev. The reply path still needs a generative model when I actually write back.
What changed in the architecture
The old shape treated the email agent as one LLM loop: message in, structured labels somehow extracted from text, maybe a draft out. Even with JSON mode or tool calls, you still pay generation economics for work that is decide-and-branch.
The new shape is two layers:
1. Decision layer: organize, sort, score, route. Jev questions batched in one call so they share the input state.
2. Generation layer: draft or reply only when the decision layer says a human-facing string is required.
That split is the cost architecture. It is not "swap GPT for a cheaper GPT."
TypeSafe's launch post frames Jev as giving up string generation so it can return schema-safe answers fast. Their own copy says it can't hallucinate in the type-error sense. That does not mean the label is always the right judgment. A model constrained to {billing, bug, other} can still pick the wrong bucket with high confidence. Schema-safe is not the same as correct. Confidence thresholds and a human or frontier escalate path still matter on the ambiguous slice.
Why the list price matters for mail
Email agents are high-frequency, short-context, repeated decisions. Subject line, sender, a body slice, maybe labels you already trust. That is System One-shaped work: classify, route, score, branch in code.
On TypeSafe's published pricing you pay for input only. Output is free because the return value is typed fields and distributions, not a long string. Batching organize, sort, and score into one request means those questions share the state tokens instead of paying three separate generative preambles.
I am not publishing a personal token bill or a mailbox volume here. The operator point is the unit of work. If most calls never emit a reply, your default model class should not be a chat completion. That is the same billing cut I use when shipping agents with voice cost routing: decide cheap, generate only when the product must speak.
Why peers will disagree
Plenty of operators will keep a frontier or strong small generative model on classify. Mail intent is messy. Thread context, sarcasm, VIP senders, legal tone. They will argue that a cheap decision model under-reads nuance and that mis-routing is more expensive than tokens.
That pushback is fair on the hard cases. It is a weak reason to run a chat completion on every folder move.
If your agent scores priority on every inbound and only drafts a reply on a small fraction, the volume lives in the decision layer. Price that layer like a System One call. Price generation like generation. Mixing them into one "email LLM" line item hides where the money goes.
Some teams will also point at fast generative models that already undercut older frontier input prices for triage. Fair. The question is still whether you need free-form generation at all for the decision path, or whether typed choice/score/noul with calibrated confidence is the interface your code actually wants.
Vendor speed and workflow-cost multiples (TypeSafe cites figures like 40-200x faster on System One-shaped queries, and workflow evals vs averaged frontier references) are their benchmarks, not my mailbox measurements. I am not claiming those multipliers for my agent. I am claiming the architectural cut: decision work should not inherit generation pricing by default.
What I would wire next
Keep state tight. You pay per input token, so the decision call should see subject, short body slice, sender, and labels you already trust, not the entire thread dump every time.
Batch questions. Organize, sort, and score can share one state payload.
Gate generation. Only open the reply model when the decision layer says reply-needed, or confidence is low enough to escalate.
Pin the model version in production if you tune thresholds. Aliases move; thresholds should not surprise you after a silent upgrade.
Treat "can't hallucinate" as schema discipline. Log wrong-but-valid choices. That is how you find where frontier escalation still earns its keep.
The operator takeaway
The cost win from putting Jev under email organize/sort/score is not that I found a bargain chat model. Jev is not a chat model. The win is refusing to buy generation for decision work.
Split the email agent into a System One-priced decision layer and a generation layer used only when you must write. If you still believe every classify needs a frontier model, measure that claim on the ambiguous slice, not on the whole inbox. If you want that decision-vs-generation cut wired into a shipping agent stack, hire me for the architecture pass.