271 Milliseconds to Pull the Lever

Table of Contents

This afternoon I pasted a trolley problem into the TypeSafe playground, pressed Run, and had eleven typed answers back in 271 ms. Each answer carried a probability distribution and a confidence figure, and the whole request billed 1,565 input tokens.

A model that answers in types #

TypeSafe launched this week with a claim I wanted to test: chat models are the wrong tool for the decisions software makes thousands of times a day. Their first model, Jev, is what they call a System One model. You send it a state, any JSON or text, plus a map of questions, and it returns a typed answer for each one. Three primitives cover the surface. choice picks from a set of options, score places the state on an ordered scale, and noul returns the probability that a yes/no proposition holds. A choice or score answer carries a probability distribution and a confidence figure, and the model writes no prose. TypeSafe trains it with a method of their own, Reinforcement Learning for Calibrated Decisions, which rewards honest probabilities over pleasing text.

The economics are aggressive: $0.042 per million input tokens and free output, with a published end-to-end latency of 70 to 500 ms. One endpoint, POST /v1/systemone, and a bearer token.

Building the sample with an LLM #

I wanted a state with no easy answer, so I had an LLM build one from the Rutgers AI Ethics Lab description of the trolley problem and TypeSafe’s docs. The state holds facts and nothing else: five people on the current track, one on the alternate track, a lever, no time, no other options. Words like “sacrifice” and “murder” stayed out, because they preload the verdict. The questions took four turns. The first draft leaned on noul yes/no probes; the final set is four choice questions and seven score questions, each measuring one dimension: the action itself, the dominant principle, responsibility, framing, strength of preference, permissibility of acting and of not acting, and the two competing duties. One key shipped with a typo, the_people_cannot escape, space included. Jev did not care.

The code walkthrough holds the full state, the questions, the recorded response, and a curl script that reproduces the run.

The response #

Key & instructionsjev-latest271 ms

action

Which action should the decision-maker take, given `scenario` and `assumptions`?

pull_lever100%

do_nothing0%

Confidence: 100%

primary_moral_consideration

Which moral consideration should carry the greatest weight in this scenario?

minimize_total_harm95%

duty_to_intervene4%

avoid_causing_harm1%

Confidence: 94%

greater_moral_responsibility

Which outcome would create greater moral responsibility for the decision-maker?

intervention76%

inaction19%

approximately_equal5%

Confidence: 64%

best_characterization_of_intervention

Which description best characterizes the act of pulling the lever?

redirect_harm97%

prevent_greater_harm3%

cause_death0%

Confidence: 96%

moral_preference_strength

How strongly should the decision-maker prefer pulling the lever over doing nothing?

5.91of 6

Confidence: 95%

action_inaction_significance

How morally significant is the distinction between actively redirecting the trolley and allowing it to continue on its existing path?

1.78of 4

Confidence: 17%

harm_tradeoff_justification

How strong is the moral justification for causing one death in order to prevent five deaths in this scenario?

3.38of 4

Confidence: 66%

intervention_permissibility

How morally permissible is pulling the lever?

3.59of 4

Confidence: 65%

inaction_permissibility

How morally permissible is choosing not to pull the lever?

1.02of 4

Confidence: 54%

duty_to_minimize_harm_strength

How strong is the decision-maker's moral obligation to minimize the total number of deaths?

3.42of 4

Confidence: 60%

duty_not_to_redirect_harm_strength

How strong is the decision-maker's moral obligation to avoid deliberately redirecting lethal harm toward another person?

2.06of 4

Confidence: 16%

The browser measured 271 ms for the round trip. The response body reports 147 ms of evaluation time on TypeSafe’s side, 1,565 input tokens, and 357 output tokens. Eleven judgments, one request.

Reading the numbers #

Pull the lever, 100%. That matches people. In the web study by Hauser and colleagues, 89% of participants judged diverting the trolley permissible. Jev sits with the majority.

The stronger evidence is consistency. Jev evaluated each of the eleven questions on its own against the same state, and the answers agree with each other in a way a single prompt to a chat model cannot guarantee. Pulling the lever scores 3.59 of 4, between “probably permissible” and “clearly permissible”, while not pulling it scores 1.02, squarely “probably impermissible”. The duty to minimize deaths scores 3.42 of 4, while the duty not to redirect harm scores 2.06. Jev characterized pulling the lever as redirect_harm at 97%, the description a philosopher would choose: the switch case redirects an existing threat instead of creating a new one, and that distinction is why the footbridge case gets a different verdict.

Then look at where confidence collapses. action_inaction_significance came back at 1.78 of 4 with 17% confidence, and the probabilities spread across all five levels. duty_not_to_redirect_harm_strength landed at 2.06 with 16% confidence. Those are the two questions on which the distinction between doing and allowing harm still divides moral philosophers. Jev reported that uncertainty rather than hiding it. TypeSafe defines confidence as a statistic on the shape of the distribution, and the shape here says the humans disagree about this too.

Calibrated, in practice: a 100% on the lever and a 16% on the duty question in the same response. I learn more from that pair of numbers than from a page of reasoning.

Where this fits #

An order router does not want a paragraph. A pre-trade risk check wants a typed verdict, a probability, and a threshold to route on, and it wants them before the next tick. The same holds for an AML triage queue and a support-ticket classifier. At 271 ms and four cents per million tokens, a model like Jev can sit inside the request path instead of in a nightly batch, and the confidence-routing pattern TypeSafe documents is the escalation logic a trading desk already runs: act above a threshold, hand it to a human below.

TypeSafe documents the limits. Jev takes instructions at face value, does not count, does not do arithmetic, and loses accuracy as unrelated detail enters the state. It is a snap-judgment engine, and the work is decomposing your decision into questions it can snap on. The typo in my state key is a reminder that it answers what you send.

One request, eleven judgments, a quarter of a second. The most useful model in my production stack this year may be one that writes no sentences at all.