Qwen3.8-27B Benchmarks: every test where Alibaba’s 27B model beats Claude Opus 4.6 Max

Qwen3.8-27B outperforms Claude Opus 4.6 Max on 16 of the 24 benchmarks in its official model card, with its largest wins in agentic coding (SWE-bench Pro: 61.7 vs 53.4), instruction following (IFBench: 79.5 vs 62.5), and computer use (OSWorld-Verified: 84.3 vs 72.7). It still loses on pure knowledge reasoning, including Humanity’s Last Exam (30.8 vs 40.0). All figures are vendor-reported by Alibaba as of the August 14, 2026 release.

TL;DR — Key Takeaways

• Qwen3.8-27B is a dense 27-billion-parameter open-weight model (Apache 2.0) released by Alibaba on August 14, 2026, with a native 262,144-token context window and image/video input.
• It beats Claude Opus 4.6 Max on agentic work: software engineering, office tasks, instruction following, desktop and mobile control.
• It loses to Opus 4.6 Max on knowledge-heavy reasoning: GPQA Diamond, Humanity’s Last Exam, and Terminal-Bench 2.1.
• Against Muse Glimmer-30B, the other open-weight comparator, Qwen3.8-27B wins every overlapping benchmark row.
• Every number is self-reported by Alibaba; no independent reproduction existed at launch. Treat the table as a ceiling, not a floor.

What Is Qwen3.8-27B?

Qwen3.8-27B is a dense, open-weight vision-language model released by Alibaba’s Qwen team on August 14, 2026, under the Apache 2.0 license. It has 27 billion parameters (28 billion including the vision encoder), a hybrid architecture of 48 Gated DeltaNet linear-attention layers and 16 full-attention layers, and a native context window of 262,144 tokens, extensible to 1 million with YaRN. It accepts text, images, and video as input and runs on a single 24 GB GPU at 4-bit quantization.

Which Benchmarks Does Qwen3.8-27B Win Against Claude Opus 4.6 Max?

Agentic coding and knowledge work

Benchmark	Qwen3.8-27B	Opus 4.6 Max	What it measures in practice	Why it matters
SWE-bench Pro	61.7	53.4	Resolving real GitHub issues end-to-end (harder “Pro” split)	Strongest single signal for autonomous coding agents on real codebases
QwenSWEBench	79.0	63.8	Qwen’s in-house agentic software engineering suite (3-run mean, 8-hour timeout)	Directional only — vendor-designed test
CoWorkBench	70.7	68.2	Long-horizon, multi-domain office tasks (in-house)	Proxy for multi-step office automation
IFBench	79.5	62.5	Instruction following: obeying precise constraints and formats	Widest text-side gap; critical for reliable structured output in production
LiveCodeBench v6	90.3	88.8	Fresh competitive-programming problems, contamination-resistant	Measures reasoning on unseen code, not memorization
DeepSWE 1.1	42.2	not reported	Agentic coding in long sessions (Claude Code harness, 256K context)	A win by absence — weight it lightly

Computer use and vision

Benchmark	Qwen3.8-27B	Opus 4.6 Max	What it measures in practice	Why it matters
OSWorld-Verified	84.3	72.7	Operating a real desktop: clicks, apps, files, browser	The strongest result on the card (+11.6); key for RPA and desktop agents
AndroidWorld	81.9	62.0	The same, on Android devices	Core metric for mobile automation
SWE-MM	38.6	27.1	Multimodal software engineering: screenshots/designs to code	Relevant for front-end generation from mockups
ClawEval-MM (Pass@3 / Avg)	57.4 / 56.9	52.5 / 54.7	Multimodal agentic tasks; Pass@3 = success in at least 1 of 3 tries	Measures agentic robustness, not just perception
MathVision (with code interpreter)	94.6	65.5	Math from diagrams, with tool use	Caveat: asymmetric prompting inflated the gap
BabyVision (with code interpreter)	85.6	12.6	Fine-grained visual reasoning	Same fixed-prompt caveat
CharXiv RQ (with CI)	90.2	66.0	Reading scientific charts from papers	Useful for technical document analysis tools
OmniDocBench 1.5	91.1	86.6	Document parsing: OCR, tables, layout	Document intelligence and data extraction
RealWorldQA	85.9	73.9	Understanding real-world photos	Everyday spatial perception
ERQA	65.5	40.8	Embodied reasoning (physical space, robotics)	Only relevant for physical/robotic agents

How Does Qwen3.8-27B Compare to Muse Glimmer-30B?

Qwen3.8-27B beats Muse Glimmer-30B on every benchmark row where both models have a reported score. Among dense open-weight models around 30 billion parameters, there is currently no contest on official numbers.

Benchmark	Qwen3.8-27B	Muse Glimmer-30B
Terminal-Bench 2.1	73.0	51.7
SWE-bench Pro	61.7	51.2
IFBench	79.5	77.0
GPQA Diamond	89.2	83.5
Humanity’s Last Exam	30.8	22.0
OSWorld-Verified	84.3	65.9
CharXiv RQ (no CI / with CI)	83.7 / 90.2	78.8
OmniDocBench 1.5	91.1	75.8

Where Does Qwen3.8-27B Lose to Claude Opus 4.6 Max?

Qwen3.8-27B trails Opus 4.6 Max on pure knowledge reasoning and raw terminal coding. The pattern is consistent: it wins on agentic execution and computer use, and loses on encyclopedic, ambiguous reasoning.

Benchmark	Qwen3.8-27B	Opus 4.6 Max	Gap
Terminal-Bench 2.1	73.0	78.2	-5.2
NL2Repo-Bench	42.3	47.6	-5.3
GPQA Diamond	89.2	91.3	-2.1
Humanity’s Last Exam	30.8	40.0	-9.2

Are the Qwen3.8-27B Benchmark Numbers Trustworthy?

All Qwen3.8-27B benchmark scores are vendor-reported by Alibaba and had not been independently reproduced at launch. Three specific caveats apply:

1. Imported competitor scores. The SWE-bench Pro comparison uses Opus 4.6 Max’s official published score rather than re-running it in the same harness, so the two numbers are not perfectly comparable.
2. Asymmetric prompting. On MathVision and BabyVision, Qwen used a single fixed prompt while competitors received the better of two prompt variants.
3. In-house benchmarks. QwenSWEBench and CoWorkBench are designed by Qwen itself; treat them as directional signals, not proof.

Independent evaluations from third-party labs were still pending as of August 19, 2026. Until they land, read the official table as a ceiling on performance, not a floor.

What Should Builders Do With These Benchmarks?

The practical reading depends on workload type. For agentic execution — coding agents, desktop and mobile automation, document parsing — Qwen3.8-27B’s numbers, Apache 2.0 license, and 24 GB GPU footprint make it the strongest self-hostable candidate in its class as of August 2026. For knowledge-heavy, ambiguous reasoning, closed frontier models still lead. A routing architecture that sends agentic work to a self-hosted Qwen3.8-27B and escalates ambiguous knowledge tasks to a frontier API captures most of the quality at a fraction of the cost.

---

Frequently Asked Questions

Is Qwen3.8-27B better than Claude Opus 4.6 Max?

On Alibaba’s official model card, Qwen3.8-27B beats Claude Opus 4.6 Max on 16 of 24 benchmarks, mainly in agentic coding, instruction following, and computer use. Opus 4.6 Max still leads on knowledge reasoning benchmarks such as GPQA Diamond and Humanity’s Last Exam. All figures are vendor-reported.

What is Qwen3.8-27B’s SWE-bench Pro score?

Qwen3.8-27B scores 61.7 on SWE-bench Pro, compared to 53.4 for Claude Opus 4.6 Max’s official score. The comparison uses different harnesses, so treat the 8.3-point gap as approximate.

Can Qwen3.8-27B run locally?

Yes. Qwen3.8-27B is released under Apache 2.0 and runs on a single 24 GB GPU at 4-bit quantization. Full BF16 weights require roughly 55.6 GB of memory.

What is the context window of Qwen3.8-27B?

Qwen3.8-27B has a native context window of 262,144 tokens, extensible to 1 million tokens using YaRN.

Does Qwen3.8-27B support images and video?

Yes. Qwen3.8-27B is a natively multimodal vision-language model that accepts text, image, and video input.

---