Kolibri: A Sovereign Open-Weight Model

21 points by miro

pegasus

Their tech report is very interesting to read: https://aleph-alpha.com/downloads/tech-report.pdf

k749gtnc9l3w
They also link blogpost versions of many of the chapters from the announcements.

Highlights for me:
- More morphological tokeniser than typical
- Adversarial training data generation for hallucination reduction
- German treated as an almost-low-resource language; behold the skew
- As a lot of purely generated data comes from Chinese-origin models, they ask GPT-OSS-120B to grade CPC-party-line-ness of some of the generated data; apparently no specific cleaning out of USA-isms, but some training on specifically EU-values entries
- Personal information masking before any training; literal recitation refusals trained in RL
- If you mess up the sandboxing too much, you don't need a powerful model to break it (of course, I have more sympathy towards people training their first, local-sized model with apparently 8-figure budget making such mistakes; not much sympathy towards 10-figure-budget runs making same omissions while training an auto-cyberattack model)

pyfisch

Pleasantly surprised to see this, I didn't expect Aleph Alpha to release an open-weights model. It's a bit ironic that they publish a bilingual Geman-English LLM on Reunification Day and their announcement blog post is only available in English.

Right now we use Qwen models at $work, they have very impressive capabilities for their size, however their German output sounds decidedly English and contains some mistakes. Kolibri's benchmark results appear to be comparable to other similarly sized models (with the exception of TerminalBench 2.1 which Qwen 3.6/3.8 really optimized for).

So this might be a model one uses productively. I hope this isn't a one-off but Aleph Alpha continues to release open-weights models.

JulianWgs

Hope this will be available on OpenRouter soon. May be even with an EU endpoint.