Shisa DE-2: An Open Decision Model

Shisa DE-2 is an Apache 2.0 decision model built on Gemma 4 26B-A4B. It answers yes/no and multiple-choice questions with probabilities instead of text, scores 59.01 on the public Decision Index 0.3, and has a median response time of 27 ms on our own test suites.
ai
llm
shisa
classification
japanese
open-source
Author

Leonard Lin

Published

October 9, 2026

TipTL;DR

Shisa DE-2 is an open decision model based on Gemma 4 26B-A4B. Given any text, image, or structured data plus a set of questions, it returns an answer and a probability for each one.

  • Apache 2.0 and runs on stock vLLM or other inference engines without any changes required
  • 59.01 on the Decision Index 0.3 public index (DE-1: 50.19, Jev 1.13.0: 57.96, OpenAI Decisions: 47.32)
  • 27 ms median response on our internal tests, against about 200 ms for the hosted services
  • Multiple inference modes available, including reasoning for max quality
  • Trained on English and Japanese, and it reads images

Links:

A few weeks ago I trained and released Shisa DE-1 as a fun weekend project. TypeSafe had just announced Jev, a “decision engine” (a general-purpose, universal classifier), to much hype and sparse technical details. Some of the open-source AI research community responded with a bit of eye-rolling (GLiNER and friends have been around for a while), some by adapting small classifier models (many based on ModernBERT), and still others decided to adapt LLMs as decision models.

My interest was piqued most by the latter direction. LLMs are actually quite good at embedding, classification, and reranking with minimal tuning, and there is something to be said for “size being size” when it comes to capabilities and generality. ModernBERT for example is great, but has no multilingual training at all. Most of the first wave of LLM-based Jev-likes were based on smaller dense LLMs, which puzzled me a bit. Why not go bigger? Doing initial testing, I found that use of larger MoEs (with low activations) could give outsized performance at extremely competitive speed. After both quality and efficiency were taken into account, I chose Gemma 4 26B-A4B as my base:

Model Params p50 ms
Shisa DE-1 26B-A4B / 4B act 20
DeepSeek-V4.1-Flash 552B-P8B-D16B 161
Qwen3.8-Flash-Next 125B-A6B 61
Nimble-9B 9B 153
kev-4B 4B 53
GLiFormer-large 576M 32
BGE-M3 zero-shot 568M 9
Laya 421M 16
GLiNER2.5-base 194M 37
ModernCE-base 150M 9
Jev (hosted) undisclosed 223

DE-1 was a quick tune that focused on reliable readouts, calibration, and a focus on improving a handful of use cases that we at Shisa AI cared most about. It’s proven surprisingly useful for us internally, and besides the HF release, we also are hosting it for free on our Shisa API platform and have created a shisa-de SDK for easy usage.

Still, there was a lot that could be improved, so I’m happy to announce the release of our latest Shisa DE-2 decision model. It has more and broader training data to shore up weaknesses, a native 256 options per question (instead of DE-1’s native 26), validation for longer contexts, and a new (default, but optional) adaptive inference policy that enables reasoning when it’s unsure of the answer.

DE-1 DE-2
Options per question 26 256
Option order in training As written Shuffled
Training data 3,632 EN/JA samples 14,637 samples per epoch from 28 task families, EN/JA
How it is asked Once Twice, and it thinks when unsure
Decision Index 0.3, public index 50.19 59.01

On the public Decision Index 0.3 suite it scores 59.01, up from DE-1’s 50.19 and ahead of hosted Jev’s 57.96 and OpenAI Decisions’ 47.32.

DE-2 performs in the top-tier of the current public Decision Index leaderboard

Example

Here’s an example of how to talk to DE-2 via the SDK:

from shisa_de import DecisionModel

with DecisionModel.from_endpoint("http://127.0.0.1:8000/v1", model="shisa-ai/shisa-de-2") as de:
    result = de.classify(
        "WINNER! Claim your free prize now!",
        {"intent": ["spam", "ham"]},
    )
    print(result["intent"])  # spam

Under the hood there is no classification head and nothing custom on the server. The SDK renders the question and its options into a prompt, asks for one token, and reads the probabilities of the option letters. That is why it is fast: an answer is one forward pass, and Gemma 4 26B-A4B only activates 3.8B of its 25.2B parameters per token.

It also means it runs on bog-standard inference software. vllm serve shisa-ai/shisa-de-2 is the entire server setup.

Asking twice, and thinking when unsure

For DE-2 we also tested a number of inference “tricks” circulating in the community. Of the ones tested, two stood out and were worth +4.4 points on the Decision Index, with the same weights:

  1. Write the question twice. The prompt contains the input, then “Read the same input again before answering:”, then the input again. This is cheap and (surprisingly) universally effective.
  2. Think when unsure. If the top answer’s probability is under 0.7 (and <=26 options), let the model reason for up to 1,024 tokens and then read the answer again.
How each question is asked Public index Median Mean
Once 54.64 35 ms 102 ms
Twice 57.57 45 ms 169 ms
Twice, think when unsure (default) 59.01 47 ms 770 ms

Repeating the question is nearly free and helps everywhere. Thinking is a different trade: almost all of its gain is on reasoning-heavy tasks (GSM8K skill goes from 70.1 to 81.9), and it costs about 4.5 times the mean latency because the model is now generating text. The median barely moves, since most questions never trigger it (~4% of questions on Decision Index).

If you need fast and predictable, turn thinking off. It is one argument, per model load or per call:

de.classify(text, labels)                    # twice, think when unsure
de.classify(text, labels, reasoning=False)   # twice
de.classify(text, labels, reads="single")    # once

Latency here is per benchmark request (a request can hold several questions) on one RTX PRO 6000 per server, one request at a time. (For best latency, run the model at TP=2).

Evals

Decision Index

DE-2, DE-1 and OpenAI Decisions are our own runs of the public suite with the official scorer; Jev’s number is from the board on 2026-10-06.

Model Decision Index 0.3, public index
Shisa DE-2 59.01
Jev 1.13.0 57.96
Shisa DE-1 50.19
OpenAI Decisions (gpt-6-luna) 47.32

(We’ve submitted DE-2, but it isn’t on the board yet, and hasn’t been run through the private set.)

All 140,178 answers from each of our three runs are in the results dataset, with the commands to reproduce them.

Our own suites, including Japanese

Public leaderboards are nice, but a big reason we trained DE-1 was for our own use cases, a lot of which are Japanese. We ran 67 internal suites against DE-2, DE-1, hosted Jev and OpenAI Decisions. A selection:

Suite Metric DE-2 DE-1 Jev 1.13.0 OpenAI Decisions
Japanese indirect refusals AUROC 0.919 0.852 0.844 0.806
Japanese leakage v3 AUROC 0.983 0.936 0.951 0.955
Japanese nuance AUROC 0.965 0.947 0.921 0.898
Hidden text v1 accuracy 0.930 0.578 0.556 0.556
Antiscam v5 accuracy 0.844 0.745 0.734 0.740
Counting probe accuracy 0.967 0.627 0.867 0.793
Guardrails AUROC 0.979 0.996 0.975 0.956
Compaction AUROC 0.850 0.869 0.923 0.886
Agent progress v2 AUROC 0.924 0.983 0.992 0.971

On the 64 suites that all four systems answered, DE-2 has the top result on 38, Jev on 29, DE-1 on 18 and OpenAI Decisions on 8 (ties count for everyone who shares them).

Speed

Response time per case on those 64 suites, each question asked once:

DE-2 DE-1 Jev 1.13.0 OpenAI Decisions
Median 27 ms 26 ms 195 ms 201 ms
Mean 40 ms 33 ms 204 ms 229 ms
95th percentile 114 ms 74 ms 280 ms 312 ms

DE-2 and DE-1 were served locally by vLLM on one H20, while Jev and OpenAI Decisions are hosted APIs measured from our Tokyo client, with network latency included. But that’s also part of the point of having the weights: you cut out the middleman.

DE-2 is great at fast-paced decision making

Vision

We did not train the vision encoder, but the same readout works on images. DE-2 scores 80.0% on VL-RewardBench (the same as OpenAI Decisions, and ahead of the 78.3% that JEV-27B-VL reports) and 0.910 AUROC on AgentRewardBench’s screenshot trajectories.

Jian-Yang’s vision, realized

The model card has the full tables.

Try it

If you try it on something we didn’t test, I’d love to hear how it does for your use case!