Jev against local models: decisions on our own hardware

We plan to use decision models such as Jev by TypeSafe in our work. A decision model does not write text. It gets a question with fixed answers, such as “Is this line mandatory, yes or no?”, and returns a probability for each answer. Our focus is sovereign AI: the data stays on our own hardware or in the EU. According to TypeSafe’s privacy policy, Jev runs in the USA, and the policy names no location in the EU. So we want to know which alternatives exist and how far they go today. We compared Jev with four open language models that run on our own graphics card. On 8 October a classifier joined, GLiNER2.5 Decide by Fastino. On speed and cost Jev leads, the run data already shows that. So we focused on the quality of the single judgments, because that is where a local model can keep up or not.

Our test case is the project match on my website. It compares a project posting line by line with my CV. We chose it because it has many facets. It asks yes or no questions, picks one of eight areas, looks for evidence in a list and gives a level from 0 to 3, in German and in English.

The short answer: two open models come close to Jev, each in a different area. Qwen3.8-27B is the least likely to name evidence that is not in the CV. Winnow-12B is not measurably behind Jev on any of the five questions (tested statistically, explained below). That does not prove it equally good. The sample is too small for that, with ten invented pairs of posting and CV and around 140 lines per language. Jev is roughly seven to ten times faster per line, as long as the local models answer their questions one after the other, and cheaper once hardware is counted. The classifier GLiNER2.5 Decide, whose maker compares it with Jev itself, does no better on average than giving every line the most frequent answer. And so far language models acting as judges decided what counts as “right”, not a person.

The test case in five questions

For every line of the posting, the match asks five small questions with fixed answers. Here is one example from our test bench, which asks every model the same cases. A posting for a customer portal contains the line “Use AI to generate and maintain test cases from requirements”. The CV next to it belongs to an experienced test automation engineer who lists a project “Test case drafts with an LLM”. The right column shows what the reference sets for this line.

#	Question	Answers	In the example
1	Is the line a requirement?	yes or no	yes
2	Is it mandatory?	yes or no	yes
3	Which of eight areas does it belong to?	one of eight, or “none of these”	AI and LLMs
4	Which CV entry is the evidence for it?	one entry, or “none”	the project “Test case drafts with an LLM”
5	How well is it covered?	level 0 to 3	level 2

On top of that come questions about the whole posting and the whole CV. Per model and run that makes 1575 questions: five per line for 275 lines in both languages, plus 72 about the postings and 128 about the CVs.

Two terms come up often below. Evidence is the answer to question 4. If a model names an entry although no entry shows the requirement, we call that invented evidence. The level is the answer to question 5, where higher means better covered.

Who decides what is right

Every share below is agreement with a reference, a model answer sheet for each line. So far that answer sheet does not come from people but from Claude models made by Anthropic. All of them judged blind, without seeing a single model answer. On 1 October 2026, Claude Opus judged the German lines and, in a separate run, the English lines. Judges from one model family can share a blind spot. That is why Claude Sonnet and Claude Fable cross-checked the German verdict on 2 October. All three agree on 575 of 636 single questions, which is 90.4 percent.

A freshly started agent decided each of the 61 disputes. It was allowed to research on the web, had to name its sources and saw no model answer. Afterwards 48 requirements instead of 43 have no evidence, and 13 levels are one step higher. We call the result the adjudicated reference. The German numbers are measured against it, the English ones against the Opus verdict alone. This reference is not fully independent, because the adjudicators are the same models as the cross-checkers. The analysis shows no lean towards the own family, though. Fable’s lone positions won about half the time with either adjudicator, Sonnet’s rarely with either. Before we call a model good enough, a person will judge a sample.

The results

The figure shows all six models side by side on the same scales. Jev is blue, the local models are grey. Under each title it says how many lines the share is based on.

Six models, the same 136 German lines

▲ significantly better than Jev · ▼ significantly worse than Jev · no mark: no significant gap

Evidence righthigher is better116 requirements

Jev		81.0 %
Winnow-12B		85.3 %
Qwen3.8-27B		87.9 %
Shisa DE-1		79.3 %
Qwen3-32B		77.6 %
GLiNER2.5-Decide-1B		13.8 %▼, significantly worse than Jev

Evidence inventedlower is better48 requirements without evidence

Jev		25.0 %
Winnow-12B		18.8 %
Qwen3.8-27B		10.4 %▲, significantly better than Jev
Shisa DE-1		35.4 %
Qwen3-32B		33.3 %
GLiNER2.5-Decide-1B		100.0 %▼, significantly worse than Jev

Level exacthigher is better116 requirements

Jev		79.3 %
Winnow-12B		77.6 %
Qwen3.8-27B		69.0 %▼, significantly worse than Jev
Shisa DE-1		66.4 %▼, significantly worse than Jev
Qwen3-32B		47.4 %▼, significantly worse than Jev
GLiNER2.5-Decide-1B		36.2 %▼, significantly worse than Jev

Area righthigher is better116 requirements

Jev		93.1 %
Winnow-12B		87.1 %
Qwen3.8-27B		86.2 %▼, significantly worse than Jev
Shisa DE-1		85.3 %▼, significantly worse than Jev
Qwen3-32B		76.7 %▼, significantly worse than Jev
GLiNER2.5-Decide-1B		51.7 %▼, significantly worse than Jev

Mandatory righthigher is better116 requirements

Jev		93.1 %
Winnow-12B		94.0 %
Qwen3.8-27B		97.4 %
Shisa DE-1		75.9 %▼, significantly worse than Jev
Qwen3-32B		100.0 %▲, significantly better than Jev
GLiNER2.5-Decide-1B		94.0 %

Time per linelower is bettermedian over 136 lines

Jev		0.25 s
Winnow-12B		2.49 s
Qwen3.8-27B		1.76 s
Shisa DE-1		2.35 s
Qwen3-32B		1.65 s
GLiNER2.5-Decide-1B		0.26 s

The shares of Jev and the four language models as a net, plus speed. GLiNER is only in the tables. Every axis runs from 0 to 100 percent, further out is better. At a glance you see where Jev leads and where the local models have their peaks. The legend hides or shows each model, and the buttons above switch to the English lines.

Where each model is strong

Seven axes from 0 to 100 percent, further out is better. A ring marks the top value on each axis.

Speed is relative: 100 % is the fastest time. That is why speed shapes the area so strongly.

• Jev
• Winnow-12B
• Qwen3.8-27B
• Shisa DE-1
• Qwen3-32B

Top value per axis: Jev on Speed (relative), Level exact, Area and Requirement recognised. Qwen3.8-27B on No invented evidence and Evidence. Qwen3-32B on Mandatory.

Top value per axis: Jev on Speed (relative), Level exact and Area. Winnow-12B on Requirement recognised. Qwen3.8-27B on No invented evidence and Evidence. Qwen3-32B on Mandatory.

Point at or tap an axis, or reach one with the Tab key. The values then appear here.

All values as a table

German lines, adjudicated reference

Model	Speed (relative)	Level exact	Area	Requirement recognised	No invented evidence	Evidence	Mandatory	Time per line
Jev	100.0 %	79.3 %	93.1 %	88.2 %	75.0 %	81.0 %	93.1 %	0.25 s
Winnow-12B	10.0 %	77.6 %	87.1 %	86.8 %	81.2 %	85.3 %	94.0 %	2.49 s
Qwen3.8-27B	14.2 %	69.0 %	86.2 %	85.3 %	89.6 %	87.9 %	97.4 %	1.76 s
Shisa DE-1	10.6 %	66.4 %	85.3 %	85.3 %	64.6 %	79.3 %	75.9 %	2.35 s
Qwen3-32B	15.2 %	47.4 %	76.7 %	85.3 %	66.7 %	77.6 %	100.0 %	1.65 s

English lines, reference Claude Opus

Model	Speed (relative)	Level exact	Area	Requirement recognised	No invented evidence	Evidence	Mandatory	Time per line
Jev	100.0 %	74.8 %	91.6 %	88.5 %	72.7 %	79.0 %	89.1 %	0.25 s
Winnow-12B	10.1 %	70.6 %	88.2 %	89.2 %	79.5 %	81.5 %	97.5 %	2.47 s
Qwen3.8-27B	14.5 %	63.9 %	81.5 %	85.6 %	90.9 %	82.4 %	97.5 %	1.73 s
Shisa DE-1	10.7 %	63.0 %	86.6 %	85.6 %	68.2 %	81.5 %	63.0 %	2.33 s
Qwen3-32B	15.6 %	47.1 %	71.4 %	85.6 %	68.2 %	78.2 %	100.0 %	1.60 s

“Significant” in the first figure means: as in our article “Jev in practice”, we tested every gap to Jev line by line with a statistical test. The test counts only the lines where one of the two models is right and the other is wrong. If its value p is below 0.05, a difference this large is rarely pure chance. A value above that does not prove equality.

The following paragraphs compare Jev with the four language models. GLiNER has a section of its own further down.

On evidence, Qwen3.8-27B leads with 87.9 percent, ahead of Winnow-12B (85.3) and Jev (81.0). Neither gap is significant. The difference on invented evidence is clear, though. By the reference, 48 requirements have no evidence in the CV. On 12 of them Jev still names an entry, Qwen3.8-27B on 5, and on those five lines Jev names one too. This gap is significant (p = 0.016). Qwen3.8-27B misses existing evidence on 2 of 68 lines, Jev on 1.

On the level, Jev is ahead with 79.3 percent. Winnow-12B follows with 77.6, and the gap is not significant (p = 0.824). The other three are significantly behind, Qwen3.8-27B with 69.0 percent (p = 0.036). Within one level of the reference, all five are right on 95 to 100 percent. Gross misses such as 0 instead of 3 are rare.

On the area, Jev is ahead as well, with 108 of 116 requirements. Against Qwen3.8-27B, Shisa DE-1 and Qwen3-32B the gap is significant. Against Winnow-12B it is not: Jev is right on 9 lines where Winnow-12B is wrong, and the other way round on 2 (p = 0.065).

On mandatory, Qwen3-32B is significantly ahead of Jev with 100 percent (Jev 93.1). Qwen3.8-27B (97.4) and Winnow-12B (94.0) are ahead too, but not significantly. Shisa DE-1 falls well behind with 75.9 percent. Whether a line is a requirement at all, all five get right on 85 to 89 percent of the lines. Almost all disagreements are header lines such as the duration or the location.

On speed and cost, Jev leads. It needs 0.25 seconds per line against 1.65 to 2.49 seconds locally. Jev answers the five questions of a line in one request. Locally they ran one after the other on a single RTX 4090 graphics card. We did not measure whether they run faster in parallel for these questions. (Addendum, 12 October 2026: for short one-token decisions the capability test C8 now measures this, with up to 16 parallel requests and the card’s power draw, C8 findings.) At list price, 1,000 lines cost about 7 US cents with Jev. Locally the electricity alone costs roughly 7 to 11 euro cents. That is estimated from the measured compute time and the card’s maximum power draw, we did not measure the power draw itself. Hardware is not counted yet. So running locally pays off where data must not leave the house, not on price.

All German values as a table:

German lines, first run on 1 October 2026, for GLiNER on 8 October 2026, against the adjudicated reference of 2 October 2026. “Requirement recognised” counts over all 136 lines, mandatory, area, evidence and level over the 116 requirements, “evidence invented” over the 48 requirements without evidence. “Level at most one off” also counts an answer as right if it is one level away from the reference. For “evidence invented” and “time per line”, lower is better. Time per line is the median over the 136 lines. ▲ significantly better than Jev, ▼ significantly worse than Jev (p < 0.05).

The reference makes a difference here. Against the Opus verdict alone, Winnow-12B led on evidence and was significantly behind Jev on the area (p = 0.021), and every level share was 5 to 6 points lower.

The English lines

The English lines count against the Opus verdict alone, without a cross-check. The table has the same layout.

English lines, first run on 1 October 2026, for GLiNER on 8 October 2026, against the Claude Opus verdict of 1 October. 139 lines, 119 of them requirements, 44 of those without evidence. Time per line is the median over the 139 lines. ▲ significantly better than Jev, ▼ significantly worse than Jev (p < 0.05).

The English lines confirm the finding on invented evidence: Jev names an entry on 12 of 44 lines without evidence, Qwen3.8-27B on 4 (p = 0.008). In English, though, Qwen3.8-27B misses existing evidence more often, on 8 of 75 lines against 4 for Jev. On the level, Qwen3.8-27B is significantly behind Jev in English as well (p = 0.019), Winnow-12B is not (p = 0.442). Mandatory is different: in English, Winnow-12B and Qwen3.8-27B are significantly ahead of Jev, with 10 lines against 0 each (p = 0.002).

Addendum, 8 October 2026: GLiNER2.5 Decide

In September 2026 Fastino published the GLiNER2.5 Decide classifiers. In the comparison table on its model card, Jev appears under the name “JevK5”. On Fastino’s own English set for fast decisions, Decide reaches 60.2 percent there and JevK5 57.6 percent. A new model that compares itself with Jev was one we wanted to see on the same test bench.

GLiNER is not a language model. It gets a text and a list of labels and returns a probability for each label. It does not write text. We measured all three models of the series: multi-Decide with 287 million parameters for many languages, Decide with 340 million and Decide-1B with about one billion, both for English. Each question went in as a classification task of its own. The labels are the answers of the question, their descriptions the answer texts from Jev’s request.

The model forced two adaptations. GLiNER rejects round brackets in the task and the labels, because it uses them as markers itself. We replaced them with square brackets. For evidence, the answers are only references such as cv.s-x. A classifier cannot look up a reference, so each evidence answer got the text of its CV entry as its description.

There are three ways to hand over the question:

• as the task text, the way the model card shows questions over a text,
• appended to the text, the way the language models get it,
• as the task text over a shortened text that holds only the sections the question names.

We added the last two because a probe beforehand showed that the model barely reads the task text. On the model card’s own example (“The treaty was signed in Paris in 1992. It entered into force the following year”), multi-Decide gave 0.993 for yes both to “Did the treaty enter into force in 1992?” and to the same question with 1993. Decide and Decide-1B behaved the same way. Three models times three ways make nine variants, one run each on 8 October 2026 on the same RTX 4090. A classifier draws no sample when it answers. A second run should give the same result, and we did not make one.

The best of the nine variants is Decide-1B with the shortened text, and it is the sixth row in both tables. Averaged over the five line questions it reaches 53.6 percent in German and 52.3 percent in English, Jev 86.9 and 84.6 percent. Simply giving every line the reference’s most frequent answer, without asking any model, reaches 54.3 and 53.4 percent. This variant is close to Jev only on mandatory, other variants also on requirement. That works only because the reference says yes on 94 and 85 percent of the lines there.

None of the nine variants ever chooses “no evidence”. Every requirement without evidence gets an invented entry, so invented evidence stands at 100 percent. The nine variants pick the right evidence on 2 to 19 percent of the requirements. Always answering “no evidence” would be right on 41 percent in German and 37 percent in English. The total score of the match does not separate fitting from non-fitting pairs either: the three pairs built not to fit get 31 to 91 points from the GLiNER variants, 12 to 34 from Jev.

Only the area partly works. Sorting a short text into a topic is the task GLiNER is built for. With the shortened text Decide-1B reaches 51.7 percent in German and 62.2 percent in English. Jev reaches 93.1 and 91.6 percent, the four language models 71 to 88 percent. The gap to Jev is significant: in German only Jev is right on 50 lines, only GLiNER on 2 (p < 0.001).

GLiNER is fast. For a German line Decide-1B with the shortened text needs 0.26 seconds, five questions one after the other. Across all nine variants it is 0.13 to 0.41 seconds. Jev answers a line in one request in 0.25 seconds, the language models need 1.65 to 2.49 seconds. The speed is of little use as long as the answers are wrong.

How we read it: GLiNER2.5 Decide is what its model card describes. It assigns a text to a list of labels, for example by intent, topic or sentiment. The match asks something else, namely whether a second text shows what the first one asks for. That needs the two texts weighed against each other, and none of the three models does that in any of the three ways. A fine-tuned GLiNER would be a different question. Fastino offers training, and we did not test it. Fastino’s table measures a different task on its own English set, with a version of Jev we do not know. We ran jev-1.13.0. Both results can be true.

Try it yourself in the lab

The numbers above are shares over many lines. How a single decision comes about is easier to see for yourself. At bks-lab.com/en/labor/entscheidungsmodelle, the stored answers from the test bench play back. No request leaves your browser for a model. Three experiments to start with:

1. Evidence that does not exist. In the experiment “Who matches the judge?”, pick the case “Steer the integration tests and the user acceptance tests with the business units”. Guess first whether the CV has an entry for it. It does not. Jev still names an entry with 0.79. Qwen3.8-27B narrowly picks “no evidence”, with 0.41 against 0.36.
2. Certainty that is not real. In the section “How a decision is made”, pick the case “The model wants to write something else” and the model Winnow-12B. Only 0.07 of the probability sits on an answer letter. After rescaling, it still shows 0.98 for yes. Then switch to Jev and compare.
3. Draw your own line. A threshold is the probability from which code takes an answer without a human looking at it. In the experiment “Where does the code draw the line?”, guess first where it should be, then drag the line yourself. The page marks in red what code then decides wrongly. The data there comes from our article “Jev in practice”.

What follows

This already runs locally. Winnow-12B is not significantly behind Jev on any of the five questions. With 12 billion parameters it is the smallest of the four models and runs on a single graphics card. No detectable gap is not proven equality, though, and 116 requirements are too few for that. On single questions, a local model is already as good or better. Qwen3.8-27B invents evidence significantly less often than Jev, on 5 instead of 12 of 48 requirements without evidence. It misses existing evidence on 2 of 68 German lines against 1, and on 8 of 75 English lines against 4. On mandatory, Qwen3-32B is significantly ahead of Jev. Working locally, you can pick a different model per question or ask a second one as a cross-check.

This is where Jev stays ahead. Clearly on speed and cost, where it is roughly seven to ten times faster per line and cheaper once hardware is counted. On level and area it is significantly ahead of both Qwen models and Shisa DE-1, but not significantly ahead of Winnow-12B. If you need a single model for all five questions today, Jev is the best choice in this comparison. Jev is not free of errors either: on 12 of 48 requirements without evidence it still names an entry.

A person has to be able to check every piece of evidence. Jev and each of the four language models sometimes invent evidence, between 10.4 and 35.4 percent of the requirements without evidence. GLiNER does so on every one. An application therefore has to show every piece of evidence it names. That holds for Jev just as much as for a local model.

A classifier for single texts does not replace weighing two of them. GLiNER2.5 Decide sorts a text into labels. That is not enough for the match, which has to read two texts against each other. A maker’s own comparison table says little about your task, you have to measure that yourself.

Measure against a reference of your own, not against Jev. Across all 1575 questions, Winnow-12B gives the same answer as Jev on 88.6 percent, more often than any other local model. That measures similarity, not correctness. Only the reference shows that Qwen3.8-27B invents evidence less often than Jev itself.

Still open for sovereign use:

1. What counts as evidence has to be fixed, then a person judges a sample.
2. The same open models at a provider in the EU, with the cost per task.
3. What running them on our own hardware costs.
4. Calibration of the local models, that is whether a certainty of 90 percent is right in 90 percent of cases.

For the technically curious: how we measured

What Jev is

TypeSafe calls Jev a System One model, trained for calibrated decisions instead of text. Calibrated means that a certainty of 0.9 is right in nine of ten cases. TypeSafe does not publish its architecture, size or training data. This article looks only at how Jev behaves.

The models

None of the five open models is Jev. Winnow-12B and Shisa DE-1 are Gemma 4 models from Google that other teams fine-tuned for decisions with fixed answers. Winnow’s model card states that Winnow is not affiliated with TypeSafe. Qwen3.8-27B and Qwen3-32B are general language models. GLiNER2.5-Decide-1B is not a language model but a classifier without text output.

#	Model	Type	Source	Base	License	Run
1	Jev 1.13.0	decision model via API	TypeSafe	not published	commercial API	API, 5 runs
2	Winnow-12B	open language model fine-tuned for decisions	EldanRing	Gemma 4 12B	Apache 2.0	local, Q8_0
3	Qwen3.8-27B	general language model	Qwen	own model	Apache 2.0	local, Ollama tag qwen3.8:27b
4	Shisa DE-1	open language model fine-tuned for decisions	shisa-ai	Gemma 4 26B-A4B (MoE)	Apache 2.0	local, Q4_K_M
5	Qwen3-32B	general language model	Qwen	own model	Apache 2.0	local, Ollama tag qwen3:32b, earlier run on the same day
6	GLiNER2.5-Decide-1B	classifier without text output, not a language model	Fastino	Ettin encoder, about 1 billion parameters	Apache 2.0	local, gliner2 2.0.0, shortened text, run on 8 October 2026

Local means one RTX 4090 running Ollama, a program that runs open models on your own machine. Q8_0 and Q4_K_M say how strongly the weights are compressed so the model fits on the card. That is about 8 and about 5 bits per weight instead of 16. GLiNER ran on the same card without Ollama, with Fastino’s Python library gliner2.

The test bench

The test bench belongs to BKS-Lab and lives in one public repository, bks-lab/lab-bench, including Jev’s answers per question, published with TypeSafe’s permission of 10 October 2026 (usage notice). We measured under its earlier name jev-match-bench, and the measurement files are unchanged in lab-bench. Version pv1 contains four postings and eight invented CVs, each in German and English. Three postings are taken word for word from the examples of the project match, and they are invented too. They form ten pairs: four that fit, three that fit partly and three that do not fit. The questions come unchanged from the project match on my website.

The lab shows this test and three more on a page of their own, each with question, data, result and the plan for the next tests: Measured on the test bench.

How a language model can be measured against Jev

A language model continues text, but it holds a probability for every possible next word piece, and that can be read out. Each question goes to the model on its own, with the answers listed below it as A, B, C. We switch off the thinking mode and write the start of the answer ourselves, so the next thing the model writes has to be the letter. We divide the probabilities of the letters by their sum, which gives them the same shape as Jev’s answer. We call that sum the mass. If it is small, the model would rather have written something else, and the rescaled certainty misleads.

What stays unequal

The cases, the scoring and the wording of the questions are the same for all models. The bench’s method description lists 14 points that stay unequal. The main ones:

• Jev gets one request per line, the local models one per question, in a text form with a system prompt of ours.
• Jev returns its probability itself, while for the local models we read it from the 20 most likely word pieces.
• Winnow-12B and Shisa DE-1 did not run in their own format. Shisa DE-1 also ran as a 4-bit version made by a third party instead of 16 bits. We cannot separate how much of its gap on mandatory comes from that.
• According to TypeSafe, Jev was trained mainly on English, while half of our lines are German.

Every number in this article holds for the model in our test bench, with this compression, on 1 October 2026, for GLiNER on 8 October 2026.

How certain a model looks is not yet calibration. The chosen answer reaches 0.99 or more on this share of the 1575 questions: Qwen3-32B 62.7 percent, Winnow-12B 57.3, Shisa DE-1 24.0, Jev 22.8, Qwen3.8-27B 14.9. At the same time, Qwen3-32B is the weakest of all on the level. The shares are not fully comparable, because Jev rounds to two decimals.

Sources

Everything public retrieved on 1 and 2 October 2026, Fastino’s model cards on 8 October 2026.

• The test case: project match on mboiman.github.io
• Winnow-12B, model card and weights: huggingface.co/EldanRing/Winnow-12B, server: github.com/EldanRing/winnow-inference
• Qwen3.8-27B: huggingface.co/Qwen/Qwen3.8-27B, in Ollama: ollama.com/library/qwen3.8
• Qwen3-32B: huggingface.co/Qwen/Qwen3-32B
• Shisa DE-1: huggingface.co/shisa-ai/shisa-de-1, quantized: huggingface.co/mradermacher/shisa-de-1-GGUF
• Jev, TypeSafe’s launch post: typesafe.ai/blog/introducing-system-one-models-and-jev, documentation: docs.typesafe.ai, privacy policy naming the USA as location: typesafe.ai/legal/privacy-policy
• GLiNER2.5 Decide, model cards with Fastino’s comparison table: huggingface.co/fastino/GLiNER2.5-Decide-1B, huggingface.co/fastino/GLiNER2.5-Decide, huggingface.co/fastino/GLiNER2.5-multi-Decide
• TypeSafe’s adapter for language models: github.com/typesafe-ai/system-one-adapter-python
• Our earlier measurements: Jev in practice, Laya against Jev

All measurements come from our runs on 1 October 2026 with the test bench jev-match-bench, version pv1, as of commit 43c39f8. Jev 1.13.0 (jev-1.13.0) ran through the TypeSafe API, the local models on an RTX 4090 with Ollama 0.30.6, Qwen3.8-27B with 0.35.0. The judges were Claude Opus (claude-opus-5-5) on 1 October, Claude Sonnet (claude-sonnet-5-5) and Claude Fable (claude-fable-5-1) on 2 October 2026. GLiNER ran on 8 October 2026 with gliner2 2.0.0 on the same RTX 4090, as of commit 2024c2c.

In the bench under results/pv1/ you find the cross-check with all 61 disputes (crosscheck.md). Next to it are the shares and p-values (score-*.md, significance-*.md), the setup with all 14 unequal points (method.md) and the agreement with Jev (report.md). Everything on GLiNER, including the eight variants not shown here, is in gliner.md, the setup in bench/run-gliner.py. The adjudicated reference is reference/pv1/adjudicated-a.de.jsonl. We recomputed time and cost per line from the run 1 files on 2 October 2026 (speed-cost.md), with Jev’s price from TypeSafe’s model overview of the same day.