I have compared instruction following of stock gpt-OSS against stock qwen and the 20B outperformed all others, intelligently following instructions and reasoning about tool calling correctly for tools it hasn’t been trained on. Additionally, it performs like a 3B model because it uses 32 experts. I don’t know where this claim that it sucks comes from but my evaluation of similarly competitive models showed it leading the pack by a lot.
The performance is great, but the censorship is ridiculous for me. I tried it as a backend for my game Guessix[1], but it would refuse for ridiculous reasons like "Cannot answer questions about copyrighted works like Harry Potter."
Iirc abliteration (ablation?) can be done without "training" and is pretty quick. It finds the individual weights related to the concept you want to ablate, and modifies those weights to "deactivate" them. Precision brain surgery, to anthropomorphize.
Do you mean like structured outputs? Unfortunately here the model is guided to explicitly tell you when you violate the rules and why, it can confuse it's system rules with the game rules and say you're not allowed to ask a question about copyrighted material etc.
Even if you're right about all of this it still doesn't refute my "fine-tuning doesn't work for 99% of customers including you" thesis.
Also, most mainstream AI benchmarks do not agree with you:
LLMarena (https://lmarena.ai/leaderboard) has GPT-OSS 120B as #53 and GPT-OSS 20B as #69 (nice), which is extremely far from leading.
DeepSeek V3.1 is ranked #9, and is a solid 60+ elo points above GPT-OSS.
I know you're going to link some of the "ya but chatbot arena sucks cus of theoretical attacks against it" paper and the llama4 debacle, but here's more evidence that GPT-OSS blows:
Qwen 32B is a dense model. The competitors for GPT-OSS 20B are max 8B dense models. You’re comparing it against models it’s not competing with and calling it crap. That’s like saying Ferrari is better than Toyota. Sure, but only if we’re comparing 0-60. If we add a budget of 20K, suddenly the Toyota starts looking more competitive.
I never claimed it was a frontier model. Just best in class for the performance it can achieve and the memory footprint it can fit in.
And btw OSS did super well on domain specific tests without fine tuning. A model I don’t need to fine tune beats one that does.
Qwen3 Next and Qwen3-30b-a3b are pretty decent proxies for GPT-OSS 120B and 20B respectively (and in fact are both MoEs with 3B active rather than 8B active parameters), and they lap GPT-OSS pretty hard in this specific benchmark, getting to #17 and #33 respectively. That being said, it's hard to take benchmarks beyond a grain of salt because real world tasks that I try to use these models for always have a lot more variation than the benchmarks illustrate. I do view GPT-OSS as a pretty good alternative to the Qwen models in some cases but there are tradeoffs -- while I see better reasoning from GPT-OSS sometimes, the prompt adherence and overall flexibility of the Qwen models makes them a lot better IMO as general purpose local open weight models.
Those models got released later than GPT-OSS. It’s like saying the Android phone released 6 months after the iPhone is faster. Maybe, but it also had 6 extra months of development.
You don't mention that MoE models horrible damage logprobs (by definition) and basically need a whole new theory of LLM sampling written for them.
Dense models are better for a reason, and the idea that "everyone is doing MoE now and dense models are dead" is total bunk nonsense.
You can quantize dense models, and 4 bit quantized Qwen 32B is still better than full precision GPT-OSS. Luckily Unsloth even gives you tools to go down to 1.58bits!
Excuse my ignorance, but what is a “dense” model? Why is comparing GPT-OSS 20B against 8B dense models “fair” but not when comparing it against 32B models?
An MoE model will not activate most of the model for any given query, so there is zero compute happening across an increasing part of the model.
Dense model means 32B parameters => 32B get used in calculation for every token. Every calculation takes time, and assuming similar latent space size (which they all have). For example Qwen-32B
MoE model has for example 80B parameters, but only 3B get used in calculation for any given token. For example Qwen3-Next 80B A3B
Performance comparison:
Qwen-32B => 56 tok/sec, 32 GB of VRAM
Qwen3-Next 80B A3B => 167 tok/sec, 85 GB of VRAM
So despite being close to 3x "bigger", Qwen3-Next is more than 3 times faster with the same compute capacity. There's a but though. But because what gets activated from one token to the next is a different subset of the model, it is still critical to have all 80B parameters loaded into memory.
So MoE performs much better with less compute, at the cost of more memory. It also performs better on benchmarks, it is a better model.
Similar techniques have long been used in ML to great success, rather than trying to create one brilliant model, create many that each have pros and cons, and then train a second model to figure out the best model for the task in front of you. There's even a name for the practice "ensemble models". MoE only kind-of fits (because you can't easily swap out models)
There's other factors, the big one being attention. That's why non-attention models, like MAMBA, will wipe the floor in terms of performance per flop (a compute unit), with anything else. When it comes to intelligence however ...
I literally talked to 5 customers last week that needed fine tuning, legitimately needed it. I get if you’re just doing basic RAG on text you generally don’t but that’s only part of the ecosystem
I didn’t say it didn’t need it, just that it needed it less. For example, previous models needed a lot of careful fine tuning to properly do tool calling. OSS does not need that.
