# EmbeddingGemma 2 vs Qwen3-Embedding (0.6B & 8B): Code Retrieval, Matryoshka Compression, and Reranking Benchmarks

When choosing an open-weight embedding model in October 2026, **Qwen3-Embedding-8B** (served on Regolo’s API at **€0.001 per request**) and **Qwen3-Embedding-0.6B** still lead pure text and multilingual retrieval, scoring **70.58** and **64.33** on MTEB Multilingual v2 against **61.36** for Google’s new **EmbeddingGemma 2**.

Where EmbeddingGemma 2 changes the equation is **code retrieval** (jumping from 68.76 to **78.68 nDCG@10** on MTEB Code v1), **native visual-document and audio embeddings**, and a modular **270M-parameter text tower** that occupies **542 MB of unified memory** on Apple Silicon versus **5.8 GB** for Qwen3-Embedding-0.6B at a 32K context window.

In our independent benchmark across 60 code, Italian/English compliance, and architecture passages, pairing local EmbeddingGemma 2 with **Qwen3-Reranker-4B** via Regolo’s `/v1/rerank` API lifted overall **Recall@5 from 92.22% to 95.56%** and code **nDCG@10 from 96.51% to 98.42%**.

 BENCHMARK · 60 DOCUMENTS & 30 QUERIES (CODE, IT↔EN, ARCHITECTURE) 

### Qwen3-Embedding-8B leads accuracy; EmbeddingGemma 2 + Reranker-4B wins on edge RAM.

Measured on October 15, 2026 across Apple M1 Ultra (64 GB unified memory, Ollama 0.40.4) and Regolo's sovereign European API (/v1/embeddings and /v1/rerank).

 Regolo API / Recommended configuration Local baseline / Unoptimized 

Overall retrieval Recall@5 (30 queries)

Qwen3-8B API98.33%

Gemma 2 + RR95.56%

Qwen3-0.6B95.00%

Gemma 2 270M92.22%

 

MTEB Multilingual v2 (published mean score)

Qwen3-8B API70.58

Qwen3-0.6B64.33

Gemma 2 270M61.36

Gemma 1 308M61.15

 

Vector index RAM for 10M chunks (Float32)

8B @ 512d MRL19.07 GB

Gemma 2 768d28.61 GB

0.6B @ 1024d38.15 GB

8B @ 4096d152.59 GB

 

Local runtime memory on Apple M1 Ultra

Regolo API0 MB local

Gemma 2 (8K)542 MB

Gemma 1 (2K)685 MB

Qwen3-0.6B (32K)5.8 GB

 

 

**8×**less vector RAM at 512d MRL
(19.07 GB vs 152.59 GB, 97.52% nDCG@10)

**10.7×**less local GPU memory for Gemma 2
(542 MB at 8K vs 5.8 GB for Qwen3-0.6B)

**+18.9**nDCG@10 points gained on Reranker-4B
when applying the full ChatML template

 

Measured on October 15, 2026 across 60 documents and 30 queries (10 Code, 10 Italian/English Compliance, 10 Technical Docs). Local models ran on Apple M1 Ultra (64 GB) via Ollama 0.40.4; Qwen3-Embedding-8B (€0.001/request) and Qwen3-Reranker-4B (€0.01/query) ran on Regolo's European Zero Data Retention API.

 

  SOVEREIGN EUROPEAN RETRIEVAL & RERANKING 

###  Run Qwen3-Embedding-8B, Qwen3-Reranker-4B, and DeepSeek-OCR-2 in Europe with Zero Data Retention 

 Pass Matryoshka vector dimensions natively on `/v1/embeddings` (€0.001/request) and add cross-encoder precision on `/v1/rerank` (€0.01/query) inside 100% green European datacenters.

 [ Start 30-day trial → ](https://regolo.ai/pricing/?utm_source=blog&utm_medium=bento_cta&utm_campaign=embeddinggemma2-qwen3-benchmark)  OpenAI-compatible API · Live in 60s  

  ✓ 100% EU Green Datacenters   ✓ Certified ZDR  

## What Google’s EmbeddingGemma 2 model card leaves out

In the original EmbeddingGemma paper (arXiv:2509.20354), Google benchmarked its 308M model directly against `Qwen3-Embedding-0.6B`, `BGE-M3`, and `jina-embeddings-v3`. In the October 2026 **EmbeddingGemma 2** model card, external open-weight baselines were removed from the primary benchmark table, leaving only a generational comparison against EmbeddingGemma 1.

Putting the official model-card disclosures from Google and Alibaba side by side explains why:

| Specification / Benchmark | EmbeddingGemma 2 | EmbeddingGemma 1 | Qwen3-Embedding-0.6B | Qwen3-Embedding-8B (Regolo API) |
|---|---|---|---|---|
| Base architecture | Gemma 4 (modular multimodal) | Gemma 3 (text encoder) | Qwen3 foundation decoder | Qwen3 foundation decoder |
| Active parameters (text vs total) | **270M text-only** / 440M text+vision / 570M text+audio / 740M full | 308M (text-only) | 595M (text-only) | 7.57B (text-only) |
| Context window | 8,192 tokens | 2,048 tokens | **32,768 tokens** | **32,768 tokens** |
| Native vector dimension | 768 | 768 | 1,024 | 4,096 |
| MRL truncation dimensions | 512, 256, 128 | 512, 256, 128 | Custom (32 to 1,024) | Custom (32 to 4,096; native `dimensions` pass-through on Regolo) |
| **MTEB Multilingual v2** (Mean Task) | 61.36 | 61.15 | **64.33** | **70.58** |
| **MTEB Code v1** (nDCG@10) | **78.68** | 68.76 | 75.41 | **80.68** |
| **MMEB v2 VisDoc** (Visual Documents, nDCG@5) | **67.84** | Not supported | Not supported | Not supported |
| **MIEB Lite** (Image) / **MSEB** (Audio MRR@10) | 64.64 / 69.54 | Not supported | Not supported | Not supported |
| Deployment & Regolo pricing | Local (Ollama `>= 0.40.4` / HF) | Local (Ollama / HF) | Local (Ollama / HF) | **Regolo API: €0.001 / request** |

Three engineering facts stand out from the published numbers:

- **Multilingual text retrieval barely moved between EmbeddingGemma 1 and 2:** a gain of 0.21 points on MTEB Multilingual v2 (61.15 to 61.36) leaves EmbeddingGemma 2 nearly 3 points behind `Qwen3-Embedding-0.6B` (64.33) and 9.22 points behind `Qwen3-Embedding-8B` (70.58).
- **Code retrieval jumped by +9.92 points:** climbing from 68.76 to 78.68 on MTEB Code v1 brings a 270M text model within 2 points of the 8-billion-parameter `Qwen3-Embedding-8B` (80.68).
- **740M is the multimodal bundle, not the text footprint:** comparing “740M vs 600M” misstates how EmbeddingGemma 2 runs in production. If your pipeline only embeds text and code, you load the `270m-bf16-text` checkpoint and skip the vision and audio towers entirely.

 BENCHMARK · RETRIEVAL & RERANKING ON REGOLO### MTEB Multilingual v2 score

Qwen3-Embedding-8B on Regolo leads EmbeddingGemma 2 by 9.22 points.

Qwen3-8B70.58

Gemma 261.36

Published MTEB Multilingual v2 mean task scores (768d for EmbeddingGemma 2, 4096d for Qwen3-Embedding-8B; Qwen3-0.6B scores 64.33).

## Our independent benchmark: Apple M1 Ultra (64 GB) vs Regolo API

Public leaderboards average dozens of datasets with different lengths and domains. To test how these models behave on workloads European engineering teams actually deploy, we ran a controlled benchmark across **60 documents and 30 graded-relevance queries** split into three domains:

1. **Code retrieval (20 implementations, 10 queries):** Python and SQL implementations (Redis Lua token buckets, async full-jitter backoff, AST function chunkers, pgvector HNSW queries, HMAC-SHA256 constant-time verification) paired with near-miss distractors (fixed-interval retries, sliding-window text splitters, unverified JWT decoders).
2. **Italian & English compliance & cross-lingual retrieval (20 passages, 10 queries):** GDPR Article 28 Zero Data Retention, EU AI Act Article 50 watermarking, DORA ICT exit strategies, Italian Garante Privacy 21-day email metadata limits, FSE 2.0 hospital pseudonymization, and US CLOUD Act exposure. Queries mix Italian questions targeting English documents and English questions targeting Italian regulatory text.
3. **Technical & API architecture docs (20 passages, 10 queries):** KV cache mechanics, Matryoshka truncation, bi-encoder vs cross-encoder attention, 1-bit binary quantization failure modes, and MoE active-parameter memory footprints.

 ZERO DATA RETENTION 

### Get engineering notes and model releases in your inbox.

Sovereign AI inference, OpenAI-compatible, EU-hosted. Benchmarks, new models (Mistral, Qwen, GLM), and cost insights. One email, no spam.

✓

#### Thank you for subscribing!

Please check your inbox to confirm your subscription.

 

   Subscribe Free 

   I accept the [Privacy Policy](https://regolo.ai/privacy-policy) \*  

    100% European infrastructure · Zero spam · Unsubscribe anytime.

 

### Hardware, runtime trap, and instruction prefixes

We executed local models on an **Apple M1 Ultra with 64 GB of unified memory** (macOS arm64) and remote models on **Regolo’s production API** (`https://api.regolo.ai/v1`). During setup we hit two operational traps that affect anyone reproducing this stack:

- **Ollama version gate for EmbeddingGemma 2:** because EmbeddingGemma 2 is built on the Gemma 4 architecture, Ollama `0.21.x` rejects the manifest with `HTTP 412: The model you are attempting to pull requires a newer version of Ollama`. You must run **Ollama `0.40.4` or newer**, which executes `embeddinggemma-2:270m-bf16-text` via the Apple `mlx` backend.
- **The 32K KV-cache memory surprise on Qwen3-Embedding-0.6B:** while `qwen3-embedding:0.6b` is a 639 MB download on disk, Ollama allocates its full 32,768-token context window in unified memory at startup, consuming **5.8 GB of GPU memory**. By contrast, `embeddinggemma-2:270m-bf16-text` (8,192-token context) consumes **542 MB**, more than **10x less runtime memory**.

Each model was evaluated both with its **official asymmetric instruction template** and with raw unprefixed queries to measure how much accuracy you lose when you drop instruction prefixes:

- **EmbeddingGemma 2 & 1 queries:** `task: code retrieval | query: {query}` (code slice) and `task: search result | query: {query}` (multilingual and docs slices). Documents use `title: {title} | text: {text}`.
- **Qwen3-Embedding (0.6B & 8B) queries:** `Instruct: {domain_task}\nQuery:{query}`. Documents are encoded without instruction tags, as required by the Qwen3 technical report.

 BENCHMARK · RETRIEVAL & RERANKING ON REGOLO### Local GPU memory in Ollama

542 MB for EmbeddingGemma 2 (8K) vs 5.8 GB for Qwen3-0.6B (32K).

Gemma 2542 MB

Qwen3-0.6B5.8 GB

Measured on Apple M1 Ultra (Ollama 0.40.4). Qwen3-0.6B pre-allocates its full 32,768-token KV cache in unified memory.

### Full-dimension retrieval results and latency

| Model & Runtime | Dim | RAM / VRAM Footprint | Query Latency p50 / p95 | Overall nDCG@10 | Overall Recall@5 | Code nDCG@10 | Multilingual IT↔EN nDCG@10 | Tech Docs nDCG@10 |
|---|---|---|---|---|---|---|---|---|
| **EmbeddingGemma 2** (`270m-bf16-text`, Local MLX) | 768 | **542 MB** (8K ctx) | **23.9 ms** / 25.8 ms | 95.93% | 92.22% | 96.51% | 96.08% | 95.21% |
| **EmbeddingGemma 1** (`308m`, Local llama.cpp) | 768 | 685 MB (2K ctx) | 23.7 ms / 26.6 ms | 97.17% | 92.22% | 99.09% | 97.51% | 94.91% |
| **Qwen3-Embedding-0.6B** (Local llama.cpp) | 1024 | 5.8 GB (32K ctx) | 30.1 ms / 43.0 ms | 97.76% | 95.00% | 97.90% | 97.51% | **97.89%** |
| **Qwen3-Embedding-8B** (Regolo API) | 4096 | **0 MB local** (Managed EU GPU) | 300.2 ms / 415.7 ms (network RTT) | **98.27%** | **98.33%** | **99.64%** | **97.90%** | 97.27% |

**What happens if you forget instruction prefixes?** When we stripped the `Instruct:` and `task:` prefixes and sent raw queries, overall `Recall@5` dropped across every model: from **92.22% to 87.22% (-5.0 points)** on EmbeddingGemma 2, from **95.00% to 92.22% (-2.78 points)** on Qwen3-Embedding-0.6B, and from **98.33% to 93.89% (-4.44 points)** on Qwen3-Embedding-8B. On technical documentation queries specifically, omitting the Qwen3 instruction prefix reduced `Qwen3-Embedding-8B` nDCG@10 from **97.27% to 94.03%**.

## Matryoshka (MRL) compression curve: 4096d down to 128d

Both EmbeddingGemma 2 and the Qwen3-Embedding family are trained with **Matryoshka Representation Learning (MRL)**, which concentrates the most discriminative variance in the leading vector coordinates. Instead of storing all 4,096 or 768 floats per chunk, you can slice the prefix of the vector and L2-renormalize it, or let Regolo’s API do it server-side by passing `"dimensions": 512` or `"dimensions": 256` to `POST /v1/embeddings`.

We tested both client-side MRL slicing across all four checkpoints and native server-side `dimensions` pass-through on Regolo’s `Qwen3-Embedding-8B` endpoint. The server-side vectors returned by Regolo at `dimensions=512` and `dimensions=256` matched our client-side L2-normalized slices to the second decimal place:

| Vector Dimension | Raw Float32 RAM (10M Chunks) | Int8 Quantized RAM (10M Chunks) | EmbeddingGemma 2 nDCG@10 (Recall@5) | Qwen3-Embedding-0.6B nDCG@10 (Recall@5) | Qwen3-Embedding-8B Regolo API nDCG@10 (Recall@5) |
|---|---|---|---|---|---|
| **4096d** (8B native) | 152.59 GB | 38.15 GB | N/A | N/A | **98.27%** (98.33%) |
| **1024d** (0.6B native / 8B MRL) | 38.15 GB | 9.54 GB | N/A | 97.76% (95.00%) | 97.70% (96.67%) |
| **768d** (Gemma native / Qwen MRL) | 28.61 GB | 7.15 GB | 95.93% (92.22%) | 98.10% (95.00%) | 97.33% (96.67%) |
| **512d** (MRL) | **19.07 GB** | **4.77 GB** | 95.92% (90.56%) | **98.14%** (93.33%) | **97.52%** (95.00%) |
| **256d** (MRL) | **9.54 GB** | **2.38 GB** | **95.95%** (87.22%) | 97.54% (95.00%) | **97.60%** (95.00%) |
| **128d** (MRL) | 4.77 GB | 1.19 GB | 94.49% (85.56%) | 96.80% (93.33%) | 97.27% (93.33%) |

 BENCHMARK · RETRIEVAL & RERANKING ON REGOLO### RAM for 10M vectors (Float32)

19.07 GB at 512d MRL vs 152.59 GB at native 4096d.

512d MRL19.07 GB

4096d Full152.59 GB

Qwen3-Embedding-8B on Regolo API (/v1/embeddings). At 512d MRL it retains 97.52% nDCG@10 vs 98.27% at 4096d.

### The MRL sweet spot for production vector databases

- **`Qwen3-Embedding-8B` truncated to `512d` or `256d` cuts raw vector RAM by 8x to 16x** (from 152.59 GB down to 19.07 GB or 9.54 GB per 10 million chunks) while keeping `nDCG@10` above **97.5%** and `Recall@5` at **95.00%**, which still outperforms native 768d EmbeddingGemma 2 (92.22% Recall@5).
- **On EmbeddingGemma 2, `512d` is the safe floor for single-stage retrieval:** while `nDCG@10` stays flat at 256d (95.95%), top-5 recall drops from **92.22% (at 768d) to 87.22% (at 256d)** and **85.56% (at 128d)** as secondary relevant documents fall below the top-5 cutoff. Furthermore, Google’s model card notes that if you use EmbeddingGemma 2 for **visual document embeddings (MMEB VisDoc)**, truncating to 128d degrades NDCG@5 from **67.84 to 62.66 (-5.18 points)**.

Here is how to request native MRL compression directly from Regolo’s `/v1/embeddings` endpoint without shipping 4,096 floats over the wire:

```
import os
import requests

REGOLO_API_KEY = os.environ["REGOLO_API_KEY"]

def embed_qwen3_mrl(texts: list[str], dimensions: int = 512) -> list[list[float]]:
    response = requests.post(
        "https://api.regolo.ai/v1/embeddings",
        headers={
            "Authorization": f"Bearer {REGOLO_API_KEY}",
            "Content-Type": "application/json",
        },
        json={
            "model": "Qwen3-Embedding-8B",
            "input": texts,
            "dimensions": dimensions,  # Passed directly to the model; returns 512d vectors
        },
        timeout=30,
    )
    response.raise_for_status()
    data = sorted(response.json()["data"], key=lambda item: item["index"])
    return [item["embedding"] for item in data]Code language: Python (python)
```

## Two-stage retrieval: local compact embedding + Qwen3-Reranker-4B on Regolo

What if your architecture requires a fast, 542 MB local embedder like `EmbeddingGemma 2` for first-stage candidate generation, but you cannot afford its 6-point Recall@5 gap against `Qwen3-Embedding-8B`? We tested a two-stage pipeline: retrieve the top 15 candidates locally with `EmbeddingGemma 2` (or `Qwen3-Embedding-0.6B`), then rerank those 15 passages using **`Qwen3-Reranker-4B`** on Regolo’s `/v1/rerank` endpoint (**€0.01 per query**, up to 14,000 tokens per call).

### Critical production finding: the Qwen3-Reranker chat-template trap

During our benchmark, we discovered a subtle but decisive behavior in how `Qwen3-Reranker-4B` scores candidates. Because Qwen3-Reranker is trained as a causal yes/no judge with a `<think>...</think>` block, sending bare strings or only `<Instruct>` / `<Query>` / `<Document>` tags without the Qwen ChatML wrapper caused `nDCG@10` to drop to **77.98%** (as the model’s token probabilities were evaluated without the system judge header and empty thinking block).

When we wrapped the query and documents in the full Qwen3-Reranker ChatML template, top-1 relevance scores snapped from noisy `0.57` probabilities to calibrated `0.9995` vs `0.0000` separations, and overall `nDCG@10` jumped by **+18.86 points (from 77.98% to 96.84%)**:

| Pipeline Configuration | Overall nDCG@10 | Overall Recall@5 | Code nDCG@10 (Recall@5) | Multilingual IT↔EN nDCG@10 | Rerank Latency p50 |
|---|---|---|---|---|---|
| Stage 1 only: **EmbeddingGemma 2 (270M)** | 95.93% | 92.22% | 96.51% (95.0%) | 96.08% | 0 ms (23.9 ms embed) |
| Stage 1 + Stage 2: **EmbeddingGemma 2 + Qwen3-Reranker-4B** (tags only, no ChatML) | 77.98% | 69.44% | 80.36% (80.0%) | 82.61% | 345.5 ms |
| Stage 1 + Stage 2: **EmbeddingGemma 2 + Qwen3-Reranker-4B (with ChatML template)** | **96.84%** (+0.91) | **95.56%** (+3.34) | **98.42% (100.0%)** | **98.46%** (+2.38) | 332.2 ms |
| Stage 1 + Stage 2: **Qwen3-Embedding-0.6B + Qwen3-Reranker-4B (with ChatML template)** | **96.84%** | **95.56%** | **98.42% (100.0%)** | 97.54% | 324.0 ms |
| Single-stage reference: **Qwen3-Embedding-8B** (Regolo API, 4096d) | **98.27%** | **98.33%** | **99.64% (100.0%)** | 97.90% | 300.2 ms (embed only) |

 BENCHMARK · RETRIEVAL &amp; RERANKING ON REGOLO### Two-stage nDCG@10 with Reranker-4B

96.84% with the ChatML template vs 77.98% without it.

With ChatML96.84%

No ChatML77.98%

Top-15 candidates from EmbeddingGemma 2 reranked by Qwen3-Reranker-4B on Regolo (/v1/rerank). Code Recall@5 reaches 100%.

Adding `Qwen3-Reranker-4B` on top of local `EmbeddingGemma 2` brings code retrieval `Recall@5` to **100.0%** and multilingual Italian/English `nDCG@10` to **98.46%** (surpassing even standalone `Qwen3-Embedding-8B` on the multilingual slice). Below is the exact Python helper with the ChatML wrapper required for calibrated scores on `https://api.regolo.ai/v1/rerank`:

```
import os
import requests

REGOLO_API_KEY = os.environ["REGOLO_API_KEY"]

def rerank_candidates_qwen3(
    query: str,
    candidate_passages: list[str],
    task: str = "Given a technical or compliance search query, retrieve relevant passages that answer the query",
    top_n: int = 5,
) -> list[dict]:
    formatted_query = (
        "<|im_start|>system\n"
        "Judge whether the Document meets the requirements based on the Query and the Instruct provided. "
        "Note that the answer can only be \"yes\" or \"no\".<|im_end|>\n"
        f"<|im_start|>user\n<Instruct>: {task}\n<Query>: {query}\n"
    )
    formatted_docs = [
        f"<Document>: {doc}<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n"
        for doc in candidate_passages
    ]
    response = requests.post(
        "https://api.regolo.ai/v1/rerank",
        headers={
            "Authorization": f"Bearer {REGOLO_API_KEY}",
            "Content-Type": "application/json",
        },
        json={
            "model": "Qwen3-Reranker-4B",
            "query": formatted_query,
            "documents": formatted_docs,
            "top_n": top_n,
        },
        timeout=30,
    )
    response.raise_for_status()
    return response.json()["results"]Code language: JavaScript (javascript)
```

## Visual document embeddings (VisDoc) vs DeepSeek-OCR-2 + Qwen3-Embedding-8B

One genuine architectural addition in EmbeddingGemma 2 is its 440M text+vision checkpoint (`embeddinggemma-2:440m-bf16-text-vision`), which scores **67.84 NDCG@5 on MMEB v2 VisDoc** by embedding page screenshots directly into a 768-dimensional vector without running OCR first. Should you replace your OCR pipeline with direct page-image embeddings?

- **When direct visual document embeddings work:** slide decks, infographics, and visual catalogs where layout, charts, or diagrams carry the primary meaning and you want a quick coarse filter over page thumbnails.
- **When single-vector page embeddings fail:** multi-column contracts, balance sheets, medical records, and engineering datasheets. Compressing an entire A4 page of 800 words and dense tables into a single 768-float vector loses exact clause numbers, SKUs, and table cell alignment. That loss grows at 128d, where VisDoc NDCG@5 drops to **62.66**.

For regulated or dense PDF workflows, our benchmarks in [DeepSeek-OCR-2 vs GLM-OCR vs PaddleOCR](https://regolo.ai/deepseek-ocr-vs-glm-ocr-vs-paddleocr-benchmark-2026/) and our [Agentic RAG tutorial](https://regolo.ai/agentic-rag-tutorial-build-retrieval-that-reasons/) show a more reliable pattern: convert pages to structured Markdown with **`deepseek-ocr-2`** on Regolo (`€0.02/request`, preserving table and layout tags), chunk by section heading, and embed the resulting text with **`Qwen3-Embedding-8B`** at `dimensions=512`.

## Decision matrix: which retrieval stack should you run?

| Workload &amp; Constraint | Recommended Model Stack | Dimension &amp; Cost Profile |
|---|---|---|
| **Enterprise multilingual RAG &amp; long documents (up to 32K tokens)** | **`Qwen3-Embedding-8B`** via Regolo API | Use `dimensions=512` (97.52% nDCG@10, 19.07 GB RAM per 10M vectors) or `4096d` (98.27% nDCG@10) at **€0.001 / request** with Zero Data Retention. |
| **On-device / edge code search &amp; IDE agents with tight RAM (&lt; 1 GB)** | **`EmbeddingGemma 2`** (`270m-bf16-text`) | **542 MB unified memory**, 23.9 ms p50 latency on Apple Silicon, 78.68 MTEB Code v1. Keep vectors at `768d` or `512d`. |
| **Hybrid local-first index + high-precision compliance retrieval** | Local **`EmbeddingGemma 2`** (Top-15) + Regolo **`Qwen3-Reranker-4B`** (Top-5) | Lifts Recall@5 from 92.22% to **95.56%** and multilingual nDCG@10 to **98.46%** at **€0.01 / rerank query** (remember the ChatML template). |
| **Self-hosted compact text embedder with 32K context** | **`Qwen3-Embedding-0.6B`** | Beats EmbeddingGemma 2 on MTEB Multilingual v2 (64.33 vs 61.36) and our tech-docs slice (97.89% vs 95.21%), but budget **5.8 GB VRAM** if serving the full 32K context in Ollama. |

---

    OPEN-SOURCE BENCHMARK SUITE · MIT LICENSE  regolo-ai/tutorials 

###  Reproduce the entire benchmark on your own hardware 

 Clone the evaluation dataset, the local Ollama + Regolo API runner, the Matryoshka (MRL) L2-normalization tests, and the raw JSON results to test your own documents and queries.

 `corpus.json` 60 passages &amp; 30 graded queries (Code, IT↔EN, Docs) 

 `benchmark.py` Ollama MLX + Regolo /v1/embeddings &amp; /v1/rerank runner 

 `test_metrics.py` nDCG@10, Recall@5, MRR@10 &amp; MRL slicing unit tests 

 `benchmark_results.json` Complete raw latency, RAM &amp; accuracy logs 

 

  bash · quick start 

 ```
git clone https://github.com/regolo-ai/tutorials.git
cd tutorials/embeddinggemma-2-vs-qwen3-embedding
cp .env.example .env && python3 benchmark.py
```

 

 [    View code &amp; dataset on GitHub → ](https://github.com/regolo-ai/tutorials/tree/main/embeddinggemma-2-vs-qwen3-embedding) ✓ Python 3.10+ stdlib runner 

 

 

 

---

## Frequently asked questions

### Is EmbeddingGemma 2 better than Qwen3-Embedding-0.6B and Qwen3-Embedding-8B?

For multilingual and general text retrieval, Qwen3-Embedding-8B (70.58 on MTEB Multilingual v2) and Qwen3-Embedding-0.6B (64.33) outperform EmbeddingGemma 2 (61.36). EmbeddingGemma 2 is stronger when you need a compact 270M-parameter text model for code retrieval (78.68 on MTEB Code v1), native image or audio embeddings, or a 542 MB local memory footprint.

### How much RAM does EmbeddingGemma 2 use compared to Qwen3-Embedding-0.6B in Ollama?

On an Apple M1 Ultra running Ollama 0.40.4, `embeddinggemma-2:270m-bf16-text` (8,192-token context) uses 542 MB of unified memory on the MLX backend. By default, `qwen3-embedding:0.6b` pre-allocates its full 32,768-token KV cache at startup and consumes 5.8 GB of memory, which is 10.7 times more than EmbeddingGemma 2.

### Why does Ollama fail with HTTP 412 when pulling EmbeddingGemma 2?

EmbeddingGemma 2 is built on the Gemma 4 architecture, which older Ollama clients do not recognize. Running `ollama pull embeddinggemma-2` on Ollama 0.21.x fails with `412: The model you are attempting to pull requires a newer version of Ollama`. Upgrading the Ollama server and client to version 0.40.4 or newer resolves the error.

### How much vector database RAM does Matryoshka (MRL) truncation save on Qwen3-Embedding-8B?

Storing 10 million chunks as native 4,096-dimensional float32 vectors requires 152.59 GB of raw vector RAM. Passing `"dimensions": 512` to `Qwen3-Embedding-8B` on Regolo's `/v1/embeddings` endpoint reduces raw float32 RAM by 8 times to 19.07 GB (or 4.77 GB with int8 quantization) while retaining 97.52% nDCG@10 versus 98.27% at 4,096 dimensions.

### Why do Qwen3-Reranker-4B relevance scores drop without the ChatML template?

Qwen3-Reranker-4B is trained as a causal yes/no judge with an empty `<think></think>` block. Sending only `<Instruct>`, `<Query>`, and `<Document>` tags without the `<|im_start|>system` prompt and assistant thinking tags caused nDCG@10 to drop to 77.98% in our test. Adding the full ChatML wrapper restored nDCG@10 to 96.84%.

---

## Ship Private AI. Not Infrastructure.

You have the private AI App architecture, bow give it an inference layer built for production.

**Regolo** gives European teams fast, OpenAI-compatible access to Mistral, Llama, Qwen, DeepSeek, GLM, and more — with zero data retention, EU data residency, and no new SDK to learn.

Change your `base_url`. Keep your LangChain code. Start shipping.

### 🚀 [Start your 30-day free trial →](https://regolo.ai/?utm_source=blog&utm_medium=cta&utm_campaign=private-rag)

Build, test, and deploy with no infrastructure to maintain.
**No credit card. No migration project. No compromise on data control.**

### 💬 [Join the Regolo Discord →](https://discord.gg/bqGrVJHeF)

Meet builders working on private RAG, local LLMs, LangChain, Ollama, and production AI systems. Share your setup, get feedback from the community, and speak directly with the Regolo team.

### 🤝 [Talk to an AI Infrastructure Engineer →](https://regolo.ai/contact?utm_source=blog&utm_medium=cta&utm_campaign=private-rag)

Running a sensitive workload, scaling beyond a proof of concept, or assessing a managed EU inference provider? Get a tailored architecture and commercial proposal for your team.

### 📂 [Clone the GitHub repository →](https://github.com/regolo-ai/tutorials/)

Get the full implementation from this guide: ingestion scripts, ChromaDB setup, hybrid retrieval, the **30-Question RAG Floor**, evaluation examples, and deployment configuration.

> **Private AI should not require a private data center.**
> Regolo gives your team an EU-native path from local experimentation to production-grade inference.

---

### Build with Regolo

- **Discord:** [Join the community →](https://discord.gg/bqGrVJHeF)
- **GitHub:** [Explore open-source workflows →](https://github.com/regolo-ai/tutorials/)
- **X / Twitter:** [Follow @regolo\_ai →](https://x.com/regolo_ai)
- **Reddit:** [Join the community →](https://www.reddit.com/r/regolo_ai/)
- **Documentation:** [Read the API docs →](https://docs.regolo.ai)
- **Contact:** [Talk to the team →](https://regolo.ai/contact)

---

*Built with ❤️ by the Regolo team. Questions? [regolo.ai/contact](https://regolo.ai/contact)* or chat with us on [Discord](https://discord.gg/bqGrVJHeF)