# Kimi K3.1 Pro vs DeepSeek 1017 vs GLM-5.3 vs Qwen3.8-27B: reading the AA Index v5 benchmarks

When [Artificial Analysis](https://artificialanalysis.ai/leaderboards/models) replaced its fourth-generation benchmark suite with Intelligence Index v5.0.1 at the end of October 2026, every model's headline score dropped while average cost and runtime per task jumped. The updated index swaps out saturated single-turn exams for nine multi-step evaluations, including `Terminal-Bench 4.0`, `FrontierCode 2.0`, `AA-Briefcase v1.1`, and `AutomationBench-AA`. On that harder scale, [Kimi K3.1 Pro](https://huggingface.co/moonshotai/Kimi-K3.1-Pro) (2.8 trillion total parameters, 104 billion active per token) took the #1 open-weight spot at 46 points and climbed to 74.61 on [BenchLM](https://benchlm.ai/models/kimi-k3-1-pro).

### Why does a 2.8-trillion-parameter model that matches Claude Opus 5.5 on Terminal-Bench 4.0 lose to a 16B-active model on repository bug fixing and cost six times more per task?

In this guide, we compare the November 2026 numbers for **Kimi K3.1 Pro**, **DeepSeek V4.1 Flash 1017**, [GLM-5.3](https://artificialanalysis.ai/models/glm-5-3) (alongside **GLM-5.2** on our API), and [Qwen3.8-27B](https://benchlm.ai/models/qwen3-8-27b), and show how we route daily engineering workloads across our European infrastructure to avoid 36-minute waits and runaway token bills.

 BENCHMARK · ARTIFICIAL ANALYSIS V5.0.1 & BENCHLM (NOV 5, 2026) 

### Kimi K3.1 Pro leads the AA Index v5 at 46 points ($3.68/task); DeepSeek 1017 wins DeepSWE v1.1 at 77.0% for $0.60/task.

Comparing Artificial Analysis Intelligence Index v5.0.1 composite scores, DeepSWE v1.1 coding pass rates, output throughput, and weighted cost per task (verified November 5, 2026).

 Open-weight models (Kimi K3.1 Pro, DeepSeek 1017, GLM-5.3, Qwen3.8-27B) Proprietary reference (Claude Opus 5.5 max) 

AA Intelligence Index v5.0.1 (9 evaluations, points)

Opus 5.559 pts

Kimi K3.146 pts

DeepSeek V433 pts

GLM-5.332 pts

Qwen3.8-27B20 pts

 

DeepSWE v1.1 multi-turn repository coding (%)

DeepSeek V477.0%

Kimi K3.175.5%

Opus 5.574.2%

GLM-5.366.9%

Qwen3.8-27B42.5%

 

Output generation speed (tokens per second)

DeepSeek V4174 tok/s

Opus 5.598 tok/s

GLM-5.365 tok/s

Qwen3.8-27B46 tok/s

Kimi K3.133 tok/s

 

Output price per 1M tokens: Regolo Core vs pay-per-token

Regolo Core~€0.065

DeepSeek V4$1.20

Qwen3.8 PAYG$3.00

GLM-5.3 PAYG$4.40

Kimi K3.1$10.00

 

 

**~€0.065 / 1M**effective token price on our €39/mo Core plan (600M tok)
(45× to 150× lower per 1M output tokens than $3 to $10 PAYG)

**13.3m vs 36.0m**average time per AA Index v5 task
(174 tok/s on 16B-active DeepSeek vs 33 tok/s on 104B-active Kimi)

**89.75% & 99.1%**Qwen3.8-27B on ExtractBench & GLM-5.2 on tau2-bench
(served on our flat €39/mo European Core API pool)

 

Sources: Artificial Analysis Intelligence Index v5.0.1 model pages, BenchLM BenchAlign v5.8 leaderboard (1,014 tracked models, verified November 5, 2026), and moonshotai/Kimi-K3.1-Pro Hugging Face evaluation tables.

 

## Active parameters, streaming speed, and hardware footprint across the four models

Before looking at individual benchmark scores, look at how many parameters each checkpoint executes per token. Total parameters determine how much GPU memory (VRAM) you need to hold the weights, while active parameters per token determine how fast the model streams tokens and how long an agent waits between tool calls.

Kimi K3.1 Pro scales its Mixture-of-Experts (MoE) architecture to 2.8 trillion total parameters and activates 104 billion parameters on every token across 896 routed experts. Even with quantization-aware training in 4-bit MXFP4, holding 2.8 trillion weights requires roughly 1.4 TB of VRAM across a multi-node GPU cluster, and computing 104 billion active weights per token holds output throughput to 33 tokens per second.

Open-weight and proprietary benchmark ledger (November 2026)

Artificial Analysis Intelligence Index v5.0.1, BenchLM v5.8, and Moonshot / DeepSeek evaluation tables

 

 Verified Nov 5, 2026 

 | Metric | Kimi K3.1 Pro2.8T / 104B MoE | DeepSeek 1017552B / 16B MoE | GLM-5.3 / 5.2753B / 40B MoE | Qwen3.8-27B27B Dense (Core API) | Claude Opus 5.5Proprietary ref. |
|---|---|---|---|---|---|
| Global composite scoreAA Index v5.0.1 / BenchLM | 46 pts (#1 Open)BenchLM: 74.61 (#13) | 33 ptsBenchLM: 65.20 (#30) | 32 pts (GLM-5.3)BenchLM: 61.00 (#50) | 20 ptsBenchLM: 54.30 (#80) | 59 pts (#2 All)BenchLM: 86.29 (#2) |
| Repository bug fixingDeepSWE v1.1 (multi-turn) | 75.5%Claude Code harness | 77.0% (#1)Beats Opus 5.5 & Kimi | 66.9%SWE-bench Vals: 95.4% | 42.5%SWE-bench Pro: 61.7% | 74.2%SWE-bench Pro: 89.9% |
| CLI & office agentsTerminal-Bench 4.0 & OfficeQA | 65.7% TB4.0OfficeQA Pro v2: 64.4% | 42.3% TB4.0OfficeQA Pro v2: 40.4% | 41.8% TB4.0TB 2.1: 88.2% | 6.3% TB4.0TB 2.1: 73.0% | 66.4% TB4.0OfficeQA Pro v2: 62.2% |
| Schema & policy accuracyExtractBench & tau2-bench | 73.3% AutoBenchGDP.pdf w/ py: 85.7% | 66.6% AutoBenchAPEX-Agents: 69.8% | 99.1% tau2 (GLM-5.2)Toolathlon: 73.0% | 89.75% ExtractBenchBeats Opus 5.5 (86.06%) | 86.06% ExtractBenchAutoBench: 69.5% |
| Speed & task durationOutput tok/s & AA v5 min/task | 33 tok/s (36.0 min)First token: 153.5s | 174 tok/s (13.3 min)First token: 50.8s | 65 tok/s (30.7 min)First token: 96.0s | 46 tok/s (39.8 min)Low effort: 26.1 min | 98 tok/s (26.2 min)First token: 345.8s |
| Price per 1M tokensRegolo Core vs AA pay-per-token | $2.00 in / $10.00 out113M output tok on AA v5 | $0.30 in / $1.20 out174M output tok (-98% cache) | ~€0.065/1M on CoreAA PAYG: $1.40 in / $4.40 out | ~€0.065/1M on CoreAA PAYG: $0.50 in / $3.00 out | $4.00 in / $20.00 outProprietary metered API |
| Cost per AA v5 taskPay-per-token vs Regolo flat pool | $3.68 / taskMetered per-token API | $0.60 / task6.1× cheaper than Kimi | $5.01 / task (PAYG)GLM-5.2 in €39/mo Core | €39/mo flat (600M tok)vs $3.53/task on AA PAYG | $10.04 / task$4/$20 per 1M tokens |

 BENCHMARK · AA INDEX V5 & BENCHLM### Output speed (tokens/second)

Active parameters per token drive streaming speed across the 4 models

DeepSeek V4174 tok/s

GLM-5.365 tok/s

Qwen3.8-27B46 tok/s

Kimi K3.133 tok/s

Artificial Analysis throughput (TTFT: 50.8s DeepSeek 1017, 96.0s GLM-5.3, 122.7s Qwen3.8-27B xhigh, 153.5s Kimi K3.1 Pro). Sources: Artificial Analysis Index v5.0.1 (Nov 5, 2026). Sources: Artificial Analysis Index v5.0.1 & BenchLM v5.8 (verified Nov 5, 2026)

   SOVEREIGN EUROPEAN INFERENCE 

Run GLM-5.2, Qwen3.8-27B, and Brick in Europe with Zero Data Retention

Get 600 million monthly tokens on our flat Regolo Core plan (€39/mo, ~€0.065/1M tokens) or deploy any Hugging Face checkpoint on dedicated European GPUs. Switch endpoints in one line of code with full OpenAI SDK compatibility.

[Start 30-day free trial (no card required) →](https://regolo.ai/pricing/)

Free credits included · 100% green energy · ✓ EU Green Datacenters ✓ Certified [Zero Data Retention](https://regolo.ai/zero-data-retention/)

## 30-step terminal recovery vs repository bug patches and schema extraction

When an engineering team asks which open-weight model is "best for coding," the answer depends on whether the agent is fixing a bug inside an existing Git repository or recovering from broken environment state across 30 shell commands. In our earlier breakdown of [DeepSeek 1017, Muse Spark 816B, and Beam 501B](https://regolo.ai/deepseek-1017-vs-muse-spark-816b-vs-beam-501b-benchmarks/), we saw how single-turn and multi-turn harnesses split coding rankings. The Artificial Analysis Index v5.0.1 data makes that split even clearer with Kimi K3.1 Pro.

However, when the workload shifts to pure software engineering inside a codebase, the ranking reverses. On `DeepSWE v1.1`, DeepSeek V4.1 Flash 1017 scores 77.0%, edging out both Kimi K3.1 Pro (75.5%) and Claude Opus 5.5 (74.2%). And as we showed in our analysis of [where open weights beat proprietary models on BenchLM](https://regolo.ai/open-weights-vs-proprietary-benchlm-benchmarks/), smaller workhorses win on focused enterprise tasks: **Qwen3.8-27B** leads LlamaIndex's `ExtractBench` at 89.75% (ahead of Claude Opus 5.5 at 86.06%), and **GLM-5.2** hits 99.1% on Artificial Analysis's `tau2-bench` customer-operations policy benchmark.

 BENCHMARK · AA INDEX V5 & BENCHLM### DeepSWE v1.1 pass rate

Multi-turn repository bug fixing: 16B-active DeepSeek vs 104B-active Kimi

DeepSeek V477.0% (#1)

Kimi K3.175.5%

Opus 5.574.2%

GLM-5.366.9%

DeepSeek V4.1 Flash 1017 (552B total, 16B active) beats Kimi K3.1 Pro (2.8T total, 104B active) and Claude Opus 5.5 on DeepSWE v1.1 while Kimi leads Terminal-Bench 4.0 at 65.7%. Sources: Artificial Analysis Index v5.0.1 & BenchLM v5.8 (verified Nov 5, 2026)

 ## The 36-minute latency trap and hidden reasoning token bills on AA Index v5

The biggest surprise in the Artificial Analysis Intelligence Index v5.0.1 release is how much wall-clock time and hidden reasoning verbosity cost when models run at maximum thinking effort. Because reasoning models bill internal scratchpad tokens (`reasoning_content`) at the full output token rate, generating 113 million to 174 million output tokens across the suite pushes Artificial Analysis's pay-per-token cost to **$3.53 per task** on Qwen3.8-27B ($0.50 in / $3.00 out per 1M), **$3.68 per task** on Kimi K3.1 Pro ($2.00 in / $10.00 out), and **$5.01 per task** on GLM-5.3 ($1.40 in / $4.40 out), versus **$0.60 per task** on DeepSeek V4.1 Flash 1017 ($0.30 in / $1.20 out).

With our flat [Regolo Core subscription](https://regolo.ai/pricing/) (**€39/month for 600 million tokens**), the effective price drops to roughly **€0.065 per million tokens** across our Core models (including `Qwen3.8-27B` and `GLM-5.2`), cutting the per-million output token rate by 45× to 150× compared to metered pay-per-token APIs.

 BENCHMARK · AA INDEX V5 & BENCHLM### Output price per 1M tokens

Regolo Core subscription (€39/mo for 600M tok) vs AA pay-per-token rates

Regolo Core~€0.065 / 1M

DeepSeek V4$1.20 / 1M

Qwen3.8-27B$3.00 / 1M

Kimi K3.1$10.00 / 1M

Our €39/mo Core plan includes 600M tokens (~€0.065/1M on Qwen3.8-27B & GLM-5.2) vs AA pay-per-token output rates ($3.00/1M Qwen3.8, $4.40/1M GLM-5.3, $10.00/1M Kimi K3.1). Sources: Artificial Analysis Index v5.0.1 & BenchLM v5.8 (verified Nov 5, 2026)

 Latency follows the active parameter count and reasoning effort settings just as strictly. Waiting 36.0 minutes per task on Kimi K3.1 Pro (with a 153.5-second wait before the first answer token appears) or 30.7 minutes on GLM-5.3 is acceptable for an overnight evaluation batch, but it stalls interactive IDE sessions in OpenCode or Roo Code where a developer is waiting to review a diff. DeepSeek V4.1 Flash 1017 cuts that average task duration to 13.3 minutes at maximum effort (and 10.3 minutes at high effort for $0.46 per task).

 BENCHMARK · AA INDEX V5 & BENCHLM### Minutes per AA Index v5 task

Average wall-clock execution time per benchmark task (lower is faster)

DeepSeek V413.3 min

Opus 5.526.2 min

GLM-5.330.7 min

Kimi K3.136.0 min

Activating 104B parameters per token slows Kimi K3.1 Pro to 33 tok/s (153.5s TTFT) and 36.0 minutes per task, vs 174 tok/s and 13.3 minutes on 16B-active DeepSeek 1017. Sources: Artificial Analysis Index v5.0.1 & BenchLM v5.8 (verified Nov 5, 2026)

 Running **Qwen3.8-27B** at `xhigh` reasoning effort generates 128 million output tokens across the suite (122.7s time to first answer token, 39.8 minutes per task). Stepping down to `medium` or `low` effort cuts time to first answer token to 35.6 seconds and 18.3 seconds respectively, while switching to non-reasoning mode drops time to first token to 5.5 seconds for structured JSON extraction and single-pass code edits.

Similarly, stepping **GLM-5.3** from `max` to `low` effort halves task duration from 30.7 minutes to 15.5 minutes (13.0s time to first token) and lowers task cost from $5.01 to $1.57, while **GLM-5.3-Flash** (320B total, 18B active) reaches 28 points on AA Index v5 at $0.58 per task.

## How to choose between Kimi K3.1 Pro, DeepSeek 1017, GLM-5.3/5.2, and Qwen3.8-27B

No engineering team needs a 2.8-trillion-parameter model running at $3.68 and 36 minutes per task to parse an invoice, enforce a customer-support tool policy, or patch a TypeScript function. In production coding harnesses like OpenCode, Roo Code, and Kilo Code, roughly 70% to 80% of agent turns are routine file reads, schema extractions, and test checks, while only 20% to 30% require deep multi-file reasoning.

On our European infrastructure, you can run **Qwen3.8-27B** and **GLM-5.2** inside our flat €39/month Core subscription (600 million monthly tokens), pair them with our open-source [Brick semantic router](https://github.com/regolo-ai/brick-SR1) (`brick-v1-beta`) so each prompt is routed by complexity, and deploy frontier MoE checkpoints like `DeepSeek-V4.1-Flash-1017` or `Kimi-K3.1-Pro` on dedicated European GPUs through our custom models service under the same [Zero Data Retention architecture](https://regolo.ai/zero-data-retention/).

 PRODUCTION ROUTING MATRIX · NOVEMBER 2026### Which open-weight model fits your engineering workflow

Match your daily engineering task to the benchmark-proven checkpoint, latency budget, and European deployment path.

 

1. Daily activity & benchmark proof

2. Model to pick

3. Ideal customer profile (ICP)

 

Workflow 1 · Multi-turn repository bug fixing

DeepSWE v1.1 (77.0% #1) & APEX-Agents (69.8%)

Interactive PR patches, test repair, and repository refactoring without 36-minute waits

 

→

Fast 16B-Active MoE

DeepSeek 1017

 $0.60 / task 174 tok/s 13.3 min/task 

 

→

Ideal profile (ICP)

Product & platform engineering teams

6.1× lower cost and 2.7× faster task completion than Kimi K3.1 Pro

 

 

Workflow 2 · Document & JSON schema extraction

LlamaIndex ExtractBench (89.75% #1)

High-precision structured JSON extraction, multimodal charts (90.2% CharXiv), and single-GPU deployability

 

→

Regolo Core API

Qwen3.8-27B

 €39/mo flat pool 46 tok/s Apache 2.0 

 

→

Ideal profile (ICP)

SaaS backends & ETL data pipelines

Beats Claude Opus 5.5 on ExtractBench and fits on a single 80 GB GPU

 

 

Workflow 3 · Policy-bound tool calling & IDE agents

tau2-bench (99.1%) & Toolathlon-Verified (73.0%)

Customer-operations policy compliance, multi-app MCP tool calls, and daily OpenCode / Roo Code loops

 

→

Regolo Core API / MoE

GLM-5.2 & GLM-5.3

 65 tok/s 88.2% TB 2.1 EU ZDR 

 

→

Ideal profile (ICP)

FinOps & regulated European enterprises

Run GLM-5.2 on our flat €39/mo Core pool (with Brick routing) to avoid $5.01/task spikes

 

 

Workflow 4 · 30-step CLI & visual PDF research

Terminal-Bench 4.0 (65.7%) & OfficeQA v2 (64.4%)

Long-horizon shell recovery and chart-heavy PDF analysis where 36 min/task is acceptable

 

→

Dedicated Multi-Node GPU

Kimi K3.1 Pro (2.8T)

 $3.68 / task 33 tok/s 104B active 

 

→

Ideal profile (ICP)

Asynchronous R&D & financial research labs

Near-Opus 5.5 terminal and multimodal accuracy for offline batch jobs

 

 

 

Sources: Artificial Analysis Intelligence Index v5.0.1, BenchLM v5.8 (Nov 5, 2026), and official Hugging Face model cards.

**Explore related benchmarks and harness guides on Regolo:** Read our deep dive into [every benchmark where Qwen3.8-27B beats frontier proprietary models](https://regolo.ai/qwen3-8-27b-benchmarks-every-test-where-alibabas-27b-model-beats-claude-opus-4-6-max/), check our [tested harness configs for Pi, OpenCode, and Kilo Code](https://regolo.ai/harness-engineering-for-qwen3-8-27b-a-technical-guide-to-pi-opencode-and-kilo-code/), compare [open-weight vs proprietary models across 716 BenchLM evaluations](https://regolo.ai/open-weights-vs-proprietary-benchlm-benchmarks/), or inspect our full [European open-weight model catalog](https://regolo.ai/models/) and [flat-rate subscription tiers](https://regolo.ai/pricing/).

 ZERO DATA RETENTION 

Get engineering notes and model releases in your inbox.

Sovereign AI inference, OpenAI-compatible, EU-hosted. Benchmarks, new models (Mistral, Qwen, GLM), and cost insights. One email, no spam.

✓

Thank you for subscribing!

Please check your inbox to confirm your subscription.

 

   Subscribe Free 

   I accept the [Privacy Policy](https://regolo.ai/privacy-policy/) \*  

    100% European infrastructure · Zero spam · Unsubscribe anytime.

 

## Frequently asked questions

**Why did Kimi K3.1 Pro and Claude Opus 5.5 scores change on Artificial Analysis Intelligence Index v5?**
On October 27, 2026, Artificial Analysis replaced Index v4 with Intelligence Index v5.0.1, introducing nine harder, agentic evaluations including `Terminal-Bench 4.0`, `FrontierCode 2.0`, `AA-Briefcase v1.1`, and `AutomationBench-AA`. On this stricter scale, Claude Opus 5.5 (max) scores 59 points ($10.04/task) and Kimi K3.1 Pro leads open-weight models at 46 points ($3.68/task).

**Is Kimi K3.1 Pro better than DeepSeek V4.1 Flash 1017 for software engineering?**
It depends on the harness and latency budget. Kimi K3.1 Pro beats DeepSeek V4.1 Flash 1017 on `Terminal-Bench 4.0` (65.7% vs 42.3%) and scanned document QA (`OfficeQA Pro v2`, 64.4% vs 40.4%), but DeepSeek V4.1 Flash 1017 scores higher on multi-turn repository bug fixing (`DeepSWE v1.1`, 77.0% vs 75.5%), streams 5.3 times faster (174 tok/s vs 33 tok/s), and costs 6.1 times less per task ($0.60 vs $3.68).

**How much GPU memory (VRAM) does Kimi K3.1 Pro require compared to Qwen3.8-27B?**
Kimi K3.1 Pro has 2.8 trillion total parameters (104 billion active per token) and requires roughly 1.4 TB of VRAM even with its native 4-bit MXFP4 weights, meaning it must run across a multi-node GPU cluster. Qwen3.8-27B has 27 billion parameters and fits on a single 80 GB GPU in BF16 (~54 GB) or a 24 GB GPU in INT4 (~16 GB), while beating both models on LlamaIndex's `ExtractBench` (89.75%).

**How do reasoning effort settings affect Qwen3.8-27B and GLM-5.3 latency on AA Index v5?**
At maximum thinking effort (`xhigh` / `max`), Qwen3.8-27B generates 128 million output tokens (122.7s time to first answer token, 39.8 minutes per task) and GLM-5.3 generates 135 million output tokens (96.0s time to first answer token, 30.7 minutes per task, $5.01/task). Lowering effort to `low` cuts GLM-5.3 task duration in half to 15.5 minutes ($1.57/task) and reduces Qwen3.8-27B time to first token to 18.3 seconds (or 5.5 seconds in non-reasoning mode).

**How can teams run GLM-5.2 and Qwen3.8-27B in Europe with predictable monthly costs?**
We serve `GLM-5.2` and `Qwen3.8-27B` directly on our flat €39/month Core subscription (600 million monthly tokens, ~€0.065/1M tokens) inside 100% green European datacenters with certified Zero Data Retention, and support custom Hugging Face checkpoints on dedicated European GPUs.

    PRODUCTION-READY EU INFERENCE · ZERO DATA RETENTION  OpenAI-Compatible API 

Ship Private AI. Not Infrastructure.

 You have the architecture and the open-source workflow. Now give it a sovereign European inference layer built for production: fast access to **GLM-5.2, Qwen3.5, Mistral, Llama, DeepSeek**, and **Brick Complexity Pro** with zero data retention and no new SDK to learn.

 DROP-IN ENDPOINT `export OPENAI_BASE_URL="https://api.regolo.ai/v1"` Keep your existing SDK & agent setup 

 

  [    30-DAY FREE TRIAL 

Start your 30-day free trial →

Build, test, and deploy with no infrastructure to maintain. Core plans from €11.70/mo (70% off first 3 months).

 ](https://regolo.ai/pricing/)  [    OPEN-SOURCE CODE 

Clone the GitHub repository →

Get the full implementation, configuration scripts, evaluation benchmarks, and ready-to-run templates.

 ](https://github.com/regolo-ai/tutorials/)  [    BUILDER COMMUNITY 

Join the Regolo Discord →

Meet engineers building with coding agents, private RAG, local LLMs, and production open-weight models.

 ](https://discord.gg/bqGrVJHeF)  [    ARCHITECTURE REVIEW 

Talk to an AI Infrastructure Engineer →

Scaling beyond a proof of concept or running regulated EU workloads? Get a tailored architecture and commercial plan.

 ](https://regolo.ai/contacts/) 

 **Private AI should not require a private data center.** Regolo gives your team an EU-native path from local experimentation to production-grade inference. 

 [API Docs](https://docs.regolo.ai) · [GitHub](https://github.com/regolo-ai/tutorials/) · [Discord](https://discord.gg/bqGrVJHeF) · [X (@regolo\_ai)](https://x.com/regolo_ai) · [Reddit](https://www.reddit.com/r/regolo_ai/) · [Contact](https://regolo.ai/contacts/) 

Built by the Regolo team · 100% EU Sovereign GPU