On June 15, 2026, Boris Cherny, head of Claude Code at Anthropic, shifted the industry perspective on AI systems with a simple admission: “I do not prompt Claude anymore; I write loops that prompt Claude.” This operational transition from zero-shot prompt engineering to deterministic loop engineering marks the end of trusting raw LLM generation in production pipelines.

Open-loop AI agents fail because they produce an output and stop, leaving zero margin to detect factual errors. Closed-loop AI agents resolve this by integrating a self-verification loop—plan, execute, verify, and correct—that compares generated text against hard environmental evidence before completing a run.

By implementing this recursive pattern, developers can reduce baseline frontier model hallucination rates, which average 1.5% to 8.5% according to Vectara (2024), down to near-zero. Grounding outputs in structured validation, rather than chasing perfect system instructions, turns unreliable AI prototypes into predictable enterprise software.

The systemic laziness of open-loop generation
The project features: code reviewer for Enterprises with zero data retention
Getting Started
Github Code

The systemic laziness of open-loop generation

Most AI agents fail in production not because the underlying language model is weak, but because the software system surrounding it is lazy. We routinely treat frontier models like magical employees who never need their work checked, expecting a single prompt to solve highly complex tasks.

The correct closed-loop pattern: a useful closed loop does four things in order: Plan, Execute, Verify, and Correct.

The crucial detail is that the system must not treat the model’s own confidence as proof; it needs either a second independent checker or a deterministic external test such as a compiler, a unit test, a file diff, a database read-back, or direct source quotations.

The project features: code reviewer for Enterprises with zero data retention

A concrete example is the enterprise Code Reviewer loop updated for Regolo, instead of a toy demo that simply shows multiple iterations, the improved version models how a developer at a company could download the script and adapt it to an internal project.

The flow starts with a terminal menu and a branded ASCII interface for REGOLO.AI CODE REVIEWER, then asks the user to configure review pricing using token-based cost inputs for prompt and completion usage. Token pricing is the most useful enterprise default because the variable cost comes from model usage, while the local verification stages have near-zero marginal cost.

The script then runs an enterprise-style closed loop:

INGEST -> GENERATE PATCH -> VERIFY STRUCTURAL -> VERIFY SEMANTIC -> DEPLOY IF PASS

That structure matters because each stage has a different operational role. Ingestion identifies the target file, generation proposes a patch, deterministic verification checks whether the patch is even valid Python, semantic verification asks an independent checker model whether the fix is minimal and likely correct, and deployment only occurs after both gates pass.

The first check is a deterministic verification, must pass a local compile() gate before anything else happens, which prevents the system from deploying code that is syntactically invalid even if the model claims success.

The second check is maker-checker separation: the model that generates the patch is not allowed to be the sole judge of its own work; a separate checker model evaluates the candidate patch and returns a structured PASS or FAIL verdict in JSON.

The third is controlled retry with exact failure feedback. When verification fails, the loop does not just “try again”; it injects the precise failure reason into the next iteration so the next patch is constrained by observable evidence rather than vague self-correction.

The fourth is auditability. Every run writes structured artifacts such as review_audit.json, review_summary.json, STATE.md, and a live code_review_skill.md, giving engineering leaders a trail of what happened, why a patch failed, why it passed, and what rule was learned for future runs.

The fifth is safe deployment behavior: the script creates a backup before overwriting the target file and blocks execution entirely if either the deterministic or semantic gate fails.

The sixth is cost governance: because the script tracks prompt tokens, completion tokens, and estimated dollar cost per call, it is easier to reason about loop efficiency in CI/CD pipelines and compare “AI review cost” with the cost of manual review and rework.

Open-loop vs enterprise Closed-loop

Aspect	Open-loop	Enterprise Closed-loop
Definition	The system performs an action without verifying the final outcome.	The system continuously measures outcomes and automatically adjusts its behavior.
Feedback	❌ None	✅ Continuous and automatic
Learning	Static	Dynamic and continuous
Decision-making	Based only on the initial input	Based on input + outcomes + historical data + KPIs
Optimization	Manual	Automatic or semi-automatic
Context	Limited to the current request	Persistent and shared across systems
Memory	None or temporary	Persistent memory (users, enterprise knowledge, business processes)
Monitoring	Limited	End-to-end observability (metrics, logs, tracing)
Quality Evaluation	Manual	Automated (evaluation pipelines)
Error Correction	Human intervention required	Self-correcting through feedback loops
Workflow	Linear	Iterative
Automation	Task automation	End-to-end process automation
Goal	Complete a task	Continuously improve the process
Scalability	Limited	High
Governance	Minimal	Policies, auditability, compliance, and governance controls
Human-in-the-loop	Only when necessary	Configurable based on risk level and business policies
Metrics	Few or none	KPIs, business metrics, cost, quality, and ROI
Data Usage	Input → Output	Input → Decision → Action → Measurement → Improvement
AI Example	A chatbot answering a user’s question.	An AI agent that executes a task, validates the outcome, collects feedback, updates its memory, and continuously improves future executions.
Business Example	Sending an email campaign.	Send campaign → analyze open rates → re-segment audience → launch follow-up campaign → measure conversions → automatically optimize future campaigns.
Typical Technologies	LLMs, APIs, prompts, simple workflows	LLMs + persistent memory + knowledge graphs + observability + evaluation pipelines + orchestration + analytics + feedback systems

Getting Started

Implement code review loop for enterprise Code Reviewer ready

Below is the compact version of the implementation flow. It keeps the core enterprise controls — OpenAI-compatible integration, deterministic verification, independent checking, bounded retries, and auditability — without turning the tutorial into a full code dump.

┌──────────────────────────────────────────────────────────────────────┐
│                    CLOSED-LOOP PIPELINE                              │
└──────────────────────────────────────────────────────────────────────┘

  [PR / IDE / CLI Event]
          │
          ▼
  ┌───────────────────────┐
  │   Review Orchestrator  │  ← policy, workflow state, cost limits
  └──────────┬────────────┘
             │
    ┌────────┴──────────────────────────────┐
    ▼                                       ▼
  ┌──────────────────────┐    ┌─────────────────────────────┐
  │  Retrieval Layer     │    │  Operational Store           │
  │  Qdrant              │    │  audit JSON, STATE.md,       │
  │  · code_chunks       │    │  review_audit.json           │
  │  · tech_docs         │    └─────────────────────────────┘
  │  · review_memory     │
  └──────────┬───────────┘
             │  top-k candidates
             ▼
  ┌──────────────────────┐
  │  Rerank Layer        │
  │  Qwen3-Reranker-4B   │  ← cross-encoder precision filter
  └──────────┬───────────┘
             │  top-n context
             ▼
  ┌──────────────────────┐
  │  Reviewer Model      │  ← maker LLM (gpt-oss-120b default)
  │  generate_patch()    │
  └──────────┬───────────┘
             │
    ┌────────┴──────────────────┐
    ▼                           ▼
  ┌────────────────┐   ┌───────────────────────┐
  │ Gate 1         │   │ Gate 2                │
  │ Deterministic  │   │ Semantic Checker      │
  │ compile / lint │   │ checker LLM           │
  │ markdown check │   │ (gemma4-31b default)  │
  └────────┬───────┘   └──────────┬────────────┘
           │                      │
           └──────────┬───────────┘
                      │ PASS
                      ▼
           ┌──────────────────────┐
           │  EXECUTE             │  ← deploy patch to disk
           │  backup + write      │
           └──────────┬───────────┘
                      │
                      ▼
           ┌──────────────────────┐
           │  Feedback Layer      │  ← persist to Qdrant review_memory
           │  · flush lessons     │  ← append to code_review_skill.md
           │  · propose doc patch │  ← re-index for future reviews
           └──────────────────────┘Code language: JavaScript (javascript)

How the Feedback Loop Improves Reviews Over Time

Review outcome
     │
     ▼
Normalize finding → classify pattern → persist to Qdrant review_memory
     │
     ▼
Append lesson to code_review_skill.md
     │
     ▼
Next review: lesson injected into GENERATE prompt
Next review: similar outcome retrieved from review_memory
     │
     ▼
Reviewer model sees: "In a previous review of a similar file,
the fix was X. The structural gate failed because Y.Code language: Bash (bash)

Quick Start

The fastest way to get started is by using our custom branded CLI utility regolo, which automates the setup of python environments, requirements, .env file key bindings, and launches local services like Qdrant automatically.

1. Zero-Config Branded Setup & Launch

Simply run the executable in the project root:

./setup.sh

This interactive utility will:

Initialize/detect your Python 3.10+ Virtual Environment (.venv) and install dependencies automatically.
Prompt you for your REGOLO_API_KEY and write it safely into .env.
Check for Docker/Qdrant and spin up the vector database, or gracefully configure Stateless Mode if Docker isn’t running.
Offer an elegant menu to test connectivity, show services health status, or immediately launch the closed-loop code reviewer.

2. Quick Command Reference

For terminal automation or pipeline integration, the ./setup.sh utility supports CLI arguments:

./setup.sh setup   # Run non-interactive full environment/service setup
./setup.sh run     # Launch the Closed-Loop Code Reviewer pipeline directly
./setup.sh status  # Display health and availability of all project components
./setup.sh help    # Display the branded CLI usage guidelinesCode language: PHP (php)

Manual Installation (Fallback)

If you prefer to configure everything manually:

1. Prerequisites

Python 3.10+
A Regolo.ai API key
(Optional) Qdrant running locally or on cloud

2. Install

git clone https://github.com/alexgenovese/closed-loop.git
cd closed-loop
python -m venv .venv && source .venv/bin/activate
pip install -r requirements.txtCode language: PHP (php)

3. Configure

cp .env.example .env
# Edit .env and set your REGOLO_API_KEYCode language: Bash (bash)

Minimal .env:

REGOLO_API_KEY=your_api_key_here
BASE_URL=https://api.regolo.ai/v1Code language: JavaScript (javascript)

Full .env with all options:

REGOLO_API_KEY=your_api_key_here
BASE_URL=https://api.regolo.ai/v1

REGOLO_MAKER_MODEL=gpt-oss-120b
REGOLO_CHECKER_MODEL=gemma4-31b
MAX_ITERATIONS=3

# Qdrant — disable with USE_QDRANT=false if not available
QDRANT_URL=http://localhost:6333
QDRANT_API_KEY=
USE_QDRANT=true

# Reranker
RERANKER_MODEL=Qwen3-Reranker-4B
USE_RERANKER=trueCode language: PHP (php)

4. (Optional) Start Qdrant

docker run -p 6333:6333 qdrant/qdrantCode language: Bash (bash)

5. Run

python main.pyCode language: Bash (bash)

Choose option 1 to run the built-in demo on a buggy Python file, or option 2 to review any file on disk.

Running on Your Own Code

# Option 2 from the interactive menu:
python main.py
> Choose an option [1]: 2
> Enter path to Python file to review: /path/to/your/service.pyCode language: PHP (php)

Or integrate programmatically:

from pathlib import Path
from openai import OpenAI
from src.config import Settings, PricingConfig
from src.orchestrator import ReviewOrchestrator

settings = Settings.from_env()
client = OpenAI(api_key="YOUR_KEY", base_url=settings.base_url)
pricing = PricingConfig()

orchestrator = ReviewOrchestrator(
    client=client,
    pricing=pricing,
    settings=settings,
    target_file=Path("src_to_review/my_service.py"),
)
result = orchestrator.run()
print(result["status"])       # SUCCESS or FAILED_AFTER_RETRIES
print(result["usage"])        # token counts and cost
print(result["iteration_log"]) # full audit trailCode language: PHP (php)

CI/CD Integration — GitHub Actions

# .github/workflows/code-review.yml
name: Closed-Loop Code Review
on: [pull_request]

jobs:
  review:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.11"
      - run: pip install -r requirements.txt
      - name: Run closed-loop review
        env:
          REGOLO_API_KEY: ${{ secrets.REGOLO_API_KEY }}
          USE_QDRANT: "false"   # stateless mode for CI
          USE_RERANKER: "false"
        run: |
          python -c "
          from pathlib import Path
          from openai import OpenAI
          from src.config import Settings, PricingConfig
          from src.orchestrator import ReviewOrchestrator
          settings = Settings.from_env()
          client = OpenAI(api_key='${{ secrets.REGOLO_API_KEY }}', base_url=settings.base_url)
          orch = ReviewOrchestrator(client, PricingConfig(), settings, Path('src_to_review/target.py'))
          r = orch.run()
          exit(0 if r['status'] == 'SUCCESS' else 1)
          "
      - uses: actions/upload-artifact@v4
        with:
          name: review-audit
          path: review_output/
Code language: PHP (php)

Compliance & Audit Trail

Every review produces a structured JSON artifact at review_output/review_audit.json:

{
  "status": "SUCCESS",
  "iterations": 2,
  "target_file": "src_to_review/payment_processor.py",
  "verifier_reason": "Patch correctly adds missing colon and preserves all function signatures.",
  "usage": {
    "total_prompt_tokens": 1842,
    "total_completion_tokens": 387,
    "total_estimated_cost_usd": 0.006
  },
  "iteration_log": [
    {"iteration": 1, "stage": "VERIFY_STRUCTURAL", "status": "FAIL", "reason": "SyntaxError on line 14: invalid syntax"},
    {"iteration": 2, "stage": "EXECUTE", "status": "PASS", "reason": "Patch is minimal and correct."}
  ]
}Code language: JSON / JSON with Comments (json)

This artifact is consumable by SIEM, compliance dashboards, or custom audit tooling.

Qdrant Payload Schema

As defined in the ROADMAP, these payload fields are indexed for filtered search:

Field	Type	Purpose
`tenant_id`	keyword	Multi-tenant isolation
`repo_id`	keyword	Repository-scoped retrieval
`path`	text	File path filtering
`language`	keyword	Language-specific routing
`owner_team`	keyword	Ownership-aware context
`source_kind`	keyword	`code` / `doc` / `review` / `incident`
`updated_at`	float (epoch)	Freshness ranking
`content`	text	Raw chunk text for retrieval

Configuration Reference

Variable	Default	Description
`REGOLO_API_KEY`	—	Required. Regolo.ai API key
`BASE_URL`	`https://api.regolo.ai/v1`	Regolo API base URL
`REGOLO_MAKER_MODEL`	`gpt-oss-120b`	Model that generates patches
`REGOLO_CHECKER_MODEL`	`gemma4-31b`	Independent semantic verifier model
`MAX_ITERATIONS`	`3`	Max retry attempts per review
`DEFAULT_INPUT_COST_PER_1K`	`0.0020`	USD cost per 1K input tokens
`DEFAULT_OUTPUT_COST_PER_1K`	`0.0060`	USD cost per 1K output tokens
`QDRANT_URL`	`http://localhost:6333`	Qdrant instance URL
`QDRANT_API_KEY`	“	Qdrant API key (Qdrant Cloud)
`USE_QDRANT`	`true`	Enable/disable retrieval layer
`RERANKER_MODEL`	`Qwen/Qwen3-Reranker-4B`	Reranker model name
`RERANKER_ENDPOINT`	“	Override reranker URL (defaults to `BASE_URL`)
`USE_RERANKER`	`true`	Enable/disable reranking

Github Code

You can download the codes on our Github repo, just download and follow the README steps. If need help you can always reach out our team on Discord 🤙

Download the Code

Start your free 30-day trial at regolo.ai and deploy LLMs with complete privacy by design. If need help you can always reach out our team on Discord 🤙

👉 Talk with our Engineers or Start your 30 days free →

Discord – Share your thoughts
GitHub Repo – Code of blog articles ready to start
Follow Us on X @regolo_ai
Open discussion on our Subreddit Community

Built with ❤️ by the Regolo team. Questions? regolo.ai/contact or chat with us on Discord

How to Build a Closed-Loop AI Agent That Catches Its Own Hallucinations

Table of Contents