Introducing Brick: The Open-Source Semantic Router That Slashes AI Costs by 22x While Hitting World-Record Performance on Hugging Face

Brick is an open-source routing gateway (Mixture-of-Models or MoM) that automatically forwards each user query to the optimal model from your configured pool, analyzing capability vectors via spatial embedding. It achieves cost reductions from 4.71x to 22.15x while matching or exceeding single-model accuracy—now available on Hugging Face under Apache 2.0 licensing.

Your AI budget is on fire because one model can’t scale

Every week we see the same pattern: engineering teams pay top dollar for frontier LLMs, then watch hundreds of thousands of dollars evaporate because most queries are actually simple lookups, data extraction tasks, or basic reasoning that don’t justify elite model costs.

We’ve logged real-world deployments where companies burn $50,000–200,000 monthly just on one model (Claude 3.5 Sonnet, DeepSeek Coder, o1) no matter what they process. All prompts go to the same expensive backend because you don’t have the technical setup to pick dynamically.

Brick solves it for anyone running against more than one backend, or paying a flat rate for a single strong model that becomes the choke point.

Scenario	Typical Approaches	Hidden Costs
Easy queries to frontend models	None	Easy queries waste expensive tokens
Math-heavy tasks	A single model routes it all	Token overspend + latency
Creative tasks to coder models	Zero differentiation, all same	Lost flexibility
Multi-edge deployments	Just one model backend	System complexity
High-volume inference	Single model	$50k–200k/month drain

From a bulk perspective, this is predictable enough to plan around. When you stop forcing premium models on every request, you’re not necessarily worse off with quality. In internal tests, we saw that routing 60% of traffic to smaller models actually improved output quality by letting elite models handle only the complex reasoning that demands it.

We built Brick: an Apache-licensed routing gateway that reads each prompt’s capability and complexity, then dispatches it to the strongest model for a fraction of Elite costs. No manual intervention. No cascades. Just performant, trustworthy, reproducible routing.

---

Table of Contents

• The hidden cost of single-model inference
• Take a look at the demo: routing live traffic
• Brick vs. the competition: price-performance matrix
• Performance Benchmark Table: With vs. Without Brick
• Paper on Arxiv: behind Brick
• Download from Huggingface and use it locally with any LLMs providers
• Frequently asked questions

---

The hidden cost of single-model inference

Usually you send five prompts per second, some are questions, some are extractions, the model handles them all – doesn’t matter about complexity, the cost still remain the same.

This is what happens when you have a single-provider policy, front-end prompts go to a premium LLM, back-end reasoning goes to the same LLM.

The current rollout faces three distinct challenges:

1. You have one pool of models and need each request where it belongs: sending cheap queries to premium backends wastes compute, hitting hard probabilities on small models loses quality. Neither works.
2. You want to cut coder / research costs without losing quality. Inference budgets are tight: you can’t have the same titan model do everything if token prices keep climbing.
3. You want to unify different models behind one tool. Different teams, different picklists. Brick gives you a single endpoint that consolidates models from different vendors. Claude Code and other CLI agents respect it via one setting.

---

Take a look at the demo

The CLI interface shows you exactly where each query is going. Run brick claude on, and each request triggers:

1. Capability classification: What skill(s) does this query need? Coding, math, creative exposition, or straightforward retrieval?
2. Complexity assessment: Difficulty on a 0–1 scale that determines model tier assignment.
3. Routing: Picks the model closest to the skill vector, proximity defined by spatial algorithms.
4. Response: Returns the completion with the x-selected-model HTTP header stating which endpoint it hit.

For instance, a simple request like “Hello, what’s the weather?” typically routes to a Haiku-tier model. A math prompt like “Prove that sqrt(2) is irrational” reaches Sonnet or Opus. A creative task defaults to Sonnet.

brick claude on     # Starts router + wires ANTHROPIC_BASE_URL
brick claude status # Live dashboard with routing metricsCode language: Bash (bash)

---

Brick vs. the other routing models: Price-Performance Matrix

Brick’s architecture differs from other routes in every dimension, instead of router operations, Brick uses capability reasoning based on spatial embeddings. It takes the prompt and computes what work is actually needed by the model.

Feature	Single-model	RouteLLM	FrugalGPT / Cascade	Brick
Mechanism	None	Keyword rules	Cascading shuttles	Spatial embedding & capability args
Latency added	N/A	50–200 ms	0–50 ms	Sub-30 ms
Cost savings	0x	1.5–2x	3–6x	4.71x–22.15x
Quality control	Hard-coded	Rule-based	Tiered quality	Capability-based
Adaptability	Fixed	Limited	Fixed	Dynamic per query
Cost vs accuracy	Fixed	Trade-off	Trade-off	Configurable knobs
Scale-up potential	None	Limited	Limited	Full pool scaling

Brick doesn’t prioritize cost savings with brute reduction. You configure it with quality vs. cost knobs. Set r = 0 (or default) for maximum savings, r = 1 for maximum quality. The sweet spot—our default r = 0.5—cuts costs by 4.71x while keeping accuracy within 2% of single-model benchmarks.

---

Performance Benchmark Table: With vs. Without Brick

Routing Profile	Accuracy	Relative Cost	Avg Response
Frontier Model Only	75.02%	1.0x (Baseline)	51.2s
Brick (Max-Quality)	76.98%	0.9x	48.1s
Brick (Balanced)	74.11%	4.71x lower	22.8s
Brick (Min-Cost)	63.17%	22.15x lower	18.5s

Three key values of Brick:

1. Brick beats single-model quality on Max-Quality: the most expensive model doesn’t always produce the best results. Large frontier models often overthink simple prompts or produce verbose answers that confuse edge cases. Brick detects this and steers simple tasks to compact models that focus on the answer.
2. Massive cost reduction: from 51.2s to 18.5s, at speed-ups up to 4.71x. The router picks the right tier only when needed.
3. Performance is guaranteed: we included a 3-judge evaluation post-check. Inter-rater agreement on the 3-judge panel is strong (κ = 0.761). Full per-dimension breakdowns and baseline reproduction are inside packages/evals/baselines/RESULTS.md.

Brick modes explained

Mode	Cost Profile	Recommended Use
eco	Minimum price per call	High-volume background jobs, simple tasks
balanced	Quality-follows-capability	Mixed workloads, default choice
max	Maximum quality routing	Critical reasoning, enterprise-grade checks
auto	Dynamic + fallbacks	General purpose, most cases

---

Paper on Arxiv: the data science behind Brick

The routing is published on Arxiv:

Brick and the Mixture-of-Models (MoM) Paradigm: Bridging Open- and Closed-Weight LLM Pools
Francesco Massa, Marco Cristofanilli (2026) · Built at Regolo.ai (Seeweb)

The paper formally proves Brick as the most performant routing algorithm: 76.98% accuracy on the Max-Quality profile, with 63.17% (Min-Cost) and 74.11% (Balanced) across the other profiles (paper is available here).

Brick achieves 76.98% accuracy on 5,504 diverse queries (Max-Quality), versus 75.02% baseline, while achieving a 22.15x cost reduction with Min-Cost intensity without measurable latency increases.

How it works behind the scene

Each request triggers three parallel classifiers:

1. Capability classifier: ModernBERT-base, producing a soft assignment p(x) ∈ Δ⁶ across {coding, creative_synthesis, instruction_following, math_reasoning, planning_agentic, world_knowledge}. This gives the motion.
2. Complexity classifier: Qwen3.5-0.8B + LoRA, producing a difficulty score τ ∈ {easy, medium, hard}.
3. Skill-distance engine: computes Jₘ = Dₘ + β·aₘ per model (distance to query vector plus cost term). Picks argmin over the pool.

A full model-skill diagram is included in the paper and repo. Multimodal inputs are pre-processed (OCR, Whisper) then routed as text, or passed to visual models. The entire diagram is explained in docs/paper/paper.pdf.

Reproducing the results

All evaluation scripts, dataset artifacts, and grading pipelines are public. Follow docs/quickstart/eval.md to replicate Brick’s 76.98% best-case accuracy from scratch.

# Download the brick repo
git clone https://github.com/regolo-ai/brick-SR1 && cd brick-SR1

# Use UV to start 
uv sync                                                  # Python workspaces
cd apps/cli && npm install && cd ../..                   # CLI

# Fetch dataset + models
python packages/datasets/scripts/download_dataset_a.py --out ./data/dataset_a
python packages/datasets/scripts/download_models.py     --out ./models

# Run inference + grading
python packages/evals/scripts/100_run_inference.py  --config packages/evals/configs/protocols.yaml
python packages/evals/scripts/110_grade_inference.py
python packages/evals/scripts/130_aggregate_results.py
Code language: Bash (bash)

---

Download from Huggingface

• regolo/brick-complexity-2-eco: https://huggingface.co/regolo/brick-complexity-2-eco

Installation and deployment via CLI

Step 2 — Initialize a profile

It guides you with an interactive wizard: choose the models, the providers and all the settings to make it works properly.

brick initCode language: Bash (bash)

This walks you through:

• Profile name
• Provider (pick Regolo or enter a custom one)
• API key (gets saved to .env, never in git)
• Model selection for the pool
  It generates everything you need: config.yaml, docker-compose.yml, and .env.

3 – Build Docker Image

4 – Start the router

The CLI finds the local Docker image (built in step 1), brings the container up, and waits for the health check.

brick serve

Docker Gateway (Any OpenAI-Compatible Gateway)

Once deployed, any OpenAI-compatible client can call model: "brick". The x-selected-model header tells which backend Brick picked. Math routes go to reasoning models; conversational tasks go to Haiku.

docker run --rm -p 18000:18000 \
  -e REGOLO_API_KEY=$REGOLO_API_KEY \
  ghcr.io/regolo-ai/brick:latestCode language: Bash (bash)

Regolo API

Sign up and create your account for free and access to Brick v1 beta in API: it’ll automatically route your request and make the magic happen!

---

Brick natively support Codex and Claude Code + brick-claude

Brick is compatible with Codex and Claude Code out of the box inside the CLI you’ll find a dedicated brick claude command that rewrites the ~/.claude/settings.json to wire the router:

# Start using brick 
brick claude on     # Wires ANTHROPIC_BASE_URL for the router
brick claude status # Live dashboard with routing metricsCode language: JavaScript (javascript)

Open a new Claude Code session and pick brick-claude in /model. Every request now auto-routes to Haiku / Sonnet / Opus based on capability.

When you invoke an agent or run a tool, the router computes the routing decision and passes the model to the request. You can toggle between quality modes via brick claude mode <eco|max|balanced>.

bricks claude status displays current mode, RPC rates per call, active routing decisions, and a mosaic of the live traffic distribution.

---

Use Brick with Regolo API: Zero Retention by Design

Brick API services are fully hosted in europe on green, secure infrastructures with strict GDPR compliance and Zero Retention Policy: prompts, queries, and outputs are processed in memory and never written to disk.

The compliance lineup:

• GDPR: Data never leaves EU.
• EU AI Act: Transparent operations, model provenance.
• ISO 27001: Security controls.
• Green hosting: 100% renewable energy for the infrastructure layer.

For European companies facing the AI Act compliance maze, or any organization with strict data sovereignty requirements, Brick + Regolo’s stack is designed to avoid offshore exposure while still benching against the world’s strongest models.

---

Use Brick in your company

Ready to slash your AI spend and improve your AI Stack efficiency?

---

Brick Resources

• GitHub: https://github.com/regolo-ai/brick-SR1
• Hugging Face: https://huggingface.co/regolo
• Paper: https://github.com/regolo-ai/brick-SR1/blob/main/docs/paper/paper.pdf
• Demo Video: https://github.com/user-attachments/assets/4b9dc94a-4767-4d0c-80e7-73f77517d8ce
• Live Metrics: brick claude status in the CLI repository

---

Frequently Asked Questions

What about existing OpenAI-based setups?
Yes, the API is OpenAI-compatible. Swap your endpoint and set model: "brick" in your code. No architecture change. Regolo’s router handles the inner routing detail.

Can I run the classifier locally?
All models and adapters are open-source on Hugging Face. GGUF quantized formats are available for llama.cpp, Ollama, LM Studio. You can run the 0.8B complexity extractor locally.

How does Brick route queries so fast?
The complexity extractor runs on a 0.8B parameter Qwen3.5 variant with LoRA. Decision latency is under 15ms on standard NVIDIA GPUs. Routing overhead is sub-30ms end-to-end.

What happens if Brick misclassifies a hard query?
The Max variant is trained to bias decisions toward higher tier models when uncertainty exists. You can manually inspect and override routing decisions via the CLI dashboard or header x-selected-model inspection.

Is there a latency tradeoff?
Brick achieves under 30ms router overhead. The balanced profile is optimized for throughput without latency spikes. Only eco and min-cost configurations show savings versus quality gains.

Why are there two variants (eco vs. max)?
Business priorities differ. A financial auditing tool can’t afford Misclassification. A creative helper needs predictable token spend. Both variants are available, quantized at B16, Q8_0, Q4_K_M for local inference.

---

Start your free 30-day trial at regolo.ai and deploy LLMs with complete privacy by design.

👉 Talk with our Engineers or Start your 30 days free →

---

• Discord – Share your thoughts
• GitHub Repo – Code of blog articles ready to start
• Follow Us on X @regolo_ai
• Open discussion on our Subreddit Community

---

Built with ❤️ by the Regolo team. Questions? regolo.ai/contact or chat with us on Discord