We Let an LLM Play Pokémon (roguelike) Blind — Here’s What Broke First

There’s a growing cottage industry of papers showing that LLMs can play Pokémon – most test them on the original Game Boy titles — structured, well-documented, present in every training corpus since 2020. We wanted to test something different, so we forked PokéRogue, a browser-based roguelike nobody trained a model on, and wired Regolo to control every player decision in real time.

The result is one import line in src/main.ts and one ~1,300-line TypeScript module. No game engine rewrite. Just a player that thinks.

This is a fork that adds an LLM-driven enemy AI powered by Regolo (gpt-oss-120b): the enemy trainer’s move selection is decided by a reasoning model instead of the default heuristic, enabling more varied and strategic opponent behavior.

---

The Setup: why PokéRogue?

PokéRogue is an open-source, browser-playable Pokémon fan game built around roguelite mechanics. Each run is procedurally generated: the six available starter Pokémon, the opponents change, the reward items change. You play until your entire party faints — no checkpoints, no save scumming mid-run.

The party heals fully every 10 waves. That single mechanic changes the optimal strategy completely. And because PokéRogue launched in 2024 and has no dedicated competitive dataset, no LLM has been fine-tuned on its specific rules. That’s exactly why it’s interesting. We’re not testing memorization. We’re testing whether a model can reason about an unfamiliar ruleset given only the context window.

The answer, with the right prompt architecture: mostly yes. With a bad prompt: embarrassingly no.

The Fork: minimum viable surgery

The design constraint was deliberate: touch the original codebase as little as possible. The fork surface consists of exactly two files modified from upstream — one import line added to src/main.ts and a new module, src/ai-llm-loader.ts, that patches game behavior at load time.

When a Regolo API key is stored in localStorage (key: aiApiKey) alongside a model name (aiModel), the module activates. The enemy keeps the game’s built-in heuristic AI — only the player side is LLM-controlled. On first launch, a overlay prompts for credentials; models are fetched dynamically from the model-group info endpoint with no authentication required, and filtered to chat mode only.

In development, a Vite proxy forwards API calls to avoid CORS issues, production builds call the API directly.

Three UI elements are injected into the DOM: a badge in the top-left showing the Regolo logo, the active model name, and a live session timer; a party panel showing all six pokemon slots with HP bars updated in real time; and a reasoning box top centered at the top that displays the model’s one-sentence explanation for each decision.

Ten phase patches: how the LLM takes control

PokéRogue’s architecture is phase-based: every discrete game state (battle command, item selection, move learning) is a Phase class. Patching prototype methods means the LLM intercepts control at exactly the right moments without touching rendering or game logic.

LoginPhase.end sets gender to MALE and marks the intro tutorial as completed, preventing the onboarding screen from blocking auto-boot. TitlePhase.start selects Classic mode automatically and calls this.end() — no title menu rendered at all.

SelectStarterPhase.start is the first real LLM call. The model receives 6 randomly selected Pokémon names and must return {"species":["name1","name2","name3"]} — exactly 3, exact names from the list. If parsing fails after 3 attempts, 3 fallbacks are picked. Well, not exactly random — they’re drawn from the same candidate pool, just unordered.

CommandPhase.start is the core patch. Every battle turn, it serializes the full game state: active Pokémon with base stats and effective stats, all 4 available moves with PP and pre-computed type effectiveness against each visible enemy, the full party (HP percentages, movesets, levels), enemy field, weather, terrain, wave index, and available Pokéballs. The LLM returns one of four actions — fight, switch, catch, or run — with mandatory reasoning with the move.

LearnMovePhase.replaceMoveCheck handles level-up move decisions.
SelectModifierPhase.start handles post-battle item selection, running concurrently with the Pokéball reveal animation to minimize wait time. SelectTargetPhase.start handles double-battle targeting — skipped entirely if targets were already pre-set during CommandPhase. SwitchPhase.start auto-selects the first non-fainted, non-field party member on fainting.
CheckSwitchPhase.start skips the “do you want to switch?” prompt. MessageUiHandler.showPrompt and related handlers auto-advance all dialog boxes with a 1,000–1,500ms delay.

---

The prompt that actually works

The initial system prompt was four words: “decide the best action.”

The model played confidently and catastrophically — catching useless Pokémon, using immune moves, switching in a Pokémon and immediately switching it back out.

The production prompt defines 8 explicit strategic checkpoints:

1. Type effectiveness. Not inferred — pre-computed with getTypeDamageMultiplier() and injected per move. The model sees "Ember — Fire Special, power 40, acc 100%, STAB | vs enemy: 2x super effective" rather than just "Ember".
2. Physical vs Special split. Compare effective ATK against effective SPA, then match against the enemy’s weaker defense stat (DEF vs SPD). A Pokémon with ATK 120 and SPA 40 should never use Special moves, even if they’re super-effective.
3. STAB. Same-type attack bonus: 1.5x multiplier. Prefer STAB super-effective combinations when available.
4. Ability interactions. Levitate grants Ground immunity. Flash Fire grants Fire immunity. Wonder Guard blocks everything except super-effective hits. These are injected by name from the game’s ability registry.
5. Status conditions. Burn halves physical ATK. Paralysis reduces SPD to 25%. Sleep and freeze skip turns. Poison and toxic drain HP each wave. The model needs these to reason about tempo.
6. Stat stages. Each +1 ATK stage multiplies physical damage by ~1.5x. A +2 ATK Pokémon hits approximately twice as hard. Without this, the model ignores setup moves and misestimates damage.
7. Speed priority. If your Pokémon outspeeds and can KO, it takes zero damage. If it’s slower and will be KO’d, switching beats attacking. Speed is injected as a raw number, not a tier label.
8. HP economy. The roguelite constraint, full party heal every 10 waves. Every HP point depleted is a resource that doesn’t come back for potentially 9 waves. “Win this turn” is the wrong objective. “Survive to wave 10” is correct.

The full type chart — all 18 types, attacking and defending — is embedded in the system prompt as a plain-text table. After this rewrite, the model’s reasoning quality improved immediately and measurably. The first clean run produced: “Ember is 2x super effective STAB vs the Grass-type enemy, and my Charmander outspeeds — I can KO before taking damage.”

---

What broke — and why It’s instructive

Chained actions without context. The original prompt let the model decide switch in one call. The next call saw a new Pokémon on field and made a fresh decision — sometimes switching again, sometimes using a suboptimal move, occasionally switching to the same Pokémon.

Fix: when the model chooses switch, it must also specify move (the attack the incoming Pokémon will use next turn). This is stored in queuedMove and consumed without an API call, binding the two decisions into one atomic intent.

• Driving complex UI flows is fragile. The original catch implementation tried to navigate a 5-step confirmation dialog via simulated button presses. It broke on every UI state variation. The better approach: identify the single precondition that triggers the complex flow and change it. For a catch with a full party, pre-release the chosen slot programmatically before calling handleCommand(Command.BALL, ...). The catch flow then hits a party with space and proceeds normally.

• All game messages look the same to an LLM — until they block. If the message handlers aren’t patched separately, the game loop stalls silently. The model isn’t informed that it’s stuck; it just never gets the next CommandPhase. Each handler class needs its own auto-advance patch with calibrated delays (1,000ms for standard messages, 1,500ms for level-up stats).

• Retry logic without backoff accumulates latency. Three attempts with no delay between them means a timeout (120s per request) can stack to 6 minutes in a worst-case failure. The current implementation retries immediately — acceptable for a demo, wrong for production – a temperature of 0.4 keeps responses stable enough that retries usually succeed on the second attempt.

---

What this tells us about LLMs in interactive systems

This project wasn’t built to prove that Regolo models are great at Pokémon, It was built to answer a more general question: how much of an LLM’s strategy quality is intrinsic, and how much is a function of context engineering?

The answer, based on this experiment: the context window is the strategy. The model’s reasoning is bounded by what it can see. Pre-computing type multipliers and injecting the result isn’t a crutch — it’s the correct architecture. The model’s job is to reason; our job is to give it the right numbers to reason with.

Navigating a UI by driving button presses is fragile: changing the precondition that triggers the UI is the durable move. Describing a game state in natural language is lossy. Serializing structured state with annotations is not.

And the roguelite constraint is a useful mental model for production LLM systems too: optimize for surviving the run, not winning the current turn. Budget your context window the way the model should budget its HP — don’t spend it all on the first wave.

The fork is open, the prompt is readable, and the reasoning box shows you exactly why the model made each choice. Try it.

---

FAQ

What is PokéRogue?
PokéRogue is an open-source, browser-playable Pokémon fan game developed by Pagefault Games, released in 2024 under AGPL-v3. It combines Pokémon battle mechanics with roguelite structure: procedurally generated runs, stackable items, and permadeath when your full party faints.

What is Regolo AI?
Regolo is an OpenAI-compatible inference API. Chat completions use the standard /v1/chat/completions endpoint with bearer-token authentication. The PokéRogue fork uses temperature: 0.4, max_tokens: 15000, response_format: { type: "json_object" }, and low reasoning effort for speed.

Can I use any OpenAI-compatible model?
Yes. The fork targets Regolo by default, but any OpenAI-compatible provider works by changing the base URL constant in src/ai-llm-loader.ts. The system prompt and serialization are model-agnostic.

Why does the enemy still use the game’s built-in AI?
An earlier version controlled both sides with LLM calls. It doubled API costs, added latency to every turn, and didn’t improve the demonstration. The game’s heuristic AI is a competent opponent at low-to-mid wave counts. The interesting question is how the LLM performs as a player, not as both players simultaneously.

How does the model handle a full party when catching?
If the party has 6 Pokémon, the model specifies a releaseSlot (0–5, never the active Pokémon’s index). The loader pre-releases that slot before issuing the catch command, so the flow finds space without triggering the 5-step party menu dialog. Change the precondition, not the flow — that’s the cleanest solution for complex UI sequences.

What happens if the LLM returns invalid JSON?
The system retries up to 3 times per decision point. If all attempts fail, deterministic fallbacks activate: first usable move for the battle command, 3 random candidates for starter selection, skip for move learning. All failures are logged to the browser console with the [ai-llm] prefix.

---