RoyaleBench
RoyaleBench evaluates language models on 21 fixed Clash Royale positions. Models choose cards, coordinates, and timing. An independent replay verifies each outcome.
Overview
Clash Royale is more than a game; it is our team’s childhood. As players ourselves, we know how difficult this game is: resource management, exploding game states, and strategic planning with imperfect information.
We have seen LLM’s play chess and poker, but we wanted to make a more unique benchmark relevant to today’s cultural zeitgeist. That benchmark is what we have to show today–RoyaleBench.
We represented 21 unique tasks using Harbor task format, Valkyrie framework, and modal compute. We isolated agents in their own game environments and gated decisions w/ a referee that determines move legality.
Our best performing model was Opus 5.5, with a pass rate of 66.7%. The average performance was 36% across 12 different LLM models.
This benchmark highlights how LLM’s still lack understanding in complex scenarios with many variables: different cards, unique interactions, and long term planning. We hope that this benchmark serves as a goal for labs to target when developing and modernizing future models. And one day, maybe we will have an LLM capable of beating Ojas (our talented friend).
Background
Clash Royale requires managing elixir, choosing deployment positions and timing, and balancing defense with tower pressure. RoyaleBench tests these decisions in fixed mid-match continuations in a deterministic simulator, verified by independent replay.
Resource management
Decide when to spend elixir and when to reserve it.
Spatial reasoning
Choose where to place cards. Invalid positions aren't corrected.
Planning and adaptation
Defend pushes and turn surviving units into counterattacks.
Results
Each model attempts all 21 tasks once on v9@1. Incomplete play scores zero. No infrastructure errors were recorded.
| # | Model | Harness | Passed | Incomplete | Success | Cost / task | Cost / move | Agent time / task |
|---|
Results compare model-and-harness configurations. The four mini-SWE-agent runs use a 12-step limit; 33/84 attempts are incomplete. Model IDs, run settings, and per-task measurements are in the results snapshot.
Success by task type
Pass rate within each task type, with passed / total counts. The harness filter applies.
One attempt per model per task. Types use the existing task labels.
Cost and outcome
Compare inference cost per task or recorded agent move. The harness filter also applies to this chart.
Anthropic costs are harness-reported. Codex costs use recorded tokens and the saved November 15 rate card, including the lower 30-minute cache-write rate because retention is unknown. Modal-hosted costs use recorded tokens and workspace inference rates. Infrastructure and invoice adjustments are excluded.
Tasks
Each task fixes the decks, hand, elixir, tower health, units, seed, opponent settings, and scoring window. The model continues from that position.
v9@1 · 21 tasks
| Task | Capability | Objective | Passed |
|---|
Inside one task
Defend a 1-HP tower
Your left tower has 1 HP. A full-health Hog Rider is approaching. Keep all three towers alive and take zero additional tower damage over the next 16 seconds, spending at most three elixir.
- Hand
- Ice Golem (2), Skeletons (1), Hog Rider (4), Musketeer (4)
- Decision
- Inspect the text state, then choose a hand slot and coordinates, or wait.
- Clock
- Each decision advances 0.5 seconds. Thinking does not advance the clock.
Cannon is next in the draw queue, but cycling to it and playing it would exceed the total spending limit. There are no further enemy deployments during this task.
What the model receives
A text observation with its hand, next card, elixir, tower HP, visible units, recent events, and time left. The opponent's hand and elixir remain hidden. Arena graphics on this page visualize the recorded state; the evaluated models do not receive these images.
How it acts
cr # inspect; no time advances
cr play SLOT X Y # deploy at chosen coordinates
cr wait # advance one decision interval
Slots are zero-based. Coordinates use the player's frame on an 18 × 32 tile arena. Task-specific intervals are 0.5, 1, or 2 seconds.
Opus vs. reference
The model and the validated reference play the same six cards in the same order, at the same engine ticks, spending 15 elixir. The model loses its tower. The reference wins without taking additional tower damage.
This case is from the saved Opus v6@1 run, not its v9 leaderboard run. V9 retains the task's original package. The position starts with 18 seconds of overtime left, a Giant approaching a 250-HP friendly tower, and a 450-HP enemy target.
Both recordings start at the same position. Play or scrub to compare the sequences. Read model trace ↗
| Play | Card | Opus coordinates | Reference coordinates |
|---|---|---|---|
| 1 | Ice Spirit | (3.5, 11.0) | (3.5, 10.0) |
| 2 | Cannon | (7.5, 10.5) | (7.0, 11.5) |
| 3 | Fireball | (3.5, 25.5) | (3.5, 25.5) |
| 4 | Hog Rider | (3.5, 15.0) | (3.5, 15.0) |
| 5 | Log | (3.5, 24.5) · offense | (4.982, 8.623) · defense |
| 6 | Ice Spirit | (4.4, 9.0) | (4.495, 9.766) |
Coordinates are tiles in the player's frame, converted at 18,000 engine units per tile. Differing placements are shaded. Several placements change together, so this comparison does not establish one action as the sole cause.
Failure analysis
All 252 attempts pass replay integrity, with 14 rejected actions across 1,584 recorded decisions.
Of 161 failed objectives, 127 complete the scenario and 34 stop early. The examples below use independently replayed policies and controlled probes.
Defend before committing to offense
Ten models spend Fireball on a tower it cannot destroy and lose to the incoming Balloon. Fable 5.2 uses Cannon to pull the Balloon away, buying time for its Giant to finish the tower.
Tower race B · V9 · 1/12 passes
Use knockback to delay the next attack
Sol's Fireball hits the Balloon but does not delay its next attack enough to protect the tower. Changing only the aim's vertical position makes the same policy pass.
Balloon Fireball · V9 · isolated aim counterfactual
Reserve counters for later threats
Opus and Fable 5.2 spend Log on the right lane while a disclosed Goblin Barrel approaches the left tower. The passing reference handles earlier threats with other cards and saves Log for the Barrel.
Reserve the counter · V9 · 0/12 passes
Time deployments to block the Hog's path
Ten models deploy Ice Golem and Skeletons starting immediately. The passing reference waits for the Hog to advance. Moving its deployments half a second earlier, with placements unchanged, loses the tower.
Hog body-block · V9 · controlled timing shift
Methodology
-
1Observe
The referee exposes the player's text view of the fixed position.
-
2Decide
The model selects a card and coordinates, or waits. Thinking pauses time.
-
3Advance
The engine applies actions and advances the task's interval. The loop repeats.
-
4Replay
A separate verifier checks the log, ticks, decisions, and resulting state.
-
5Score
Reward is one only when integrity passes and every objective is met.
Opponent: the Planner
Planner is a deterministic search bot with a ten-second look-ahead, three plies, four top candidates, and three responses per candidate. It reads the true engine state, including both hands and elixir. Some tactical windows permit no further enemy plays.
Task construction
Tasks use the Harbor directory format, run through Valkyrie in
Modal sandboxes, and expose the protected
cr interface. A separate referee owns the engine
state and action log. The verifier replays the commands
independently; there is no model-based judge.
Scoring and variance
Score = 100 × objectives passed / 21. Spending, damage, crowns, and match outcomes are diagnostics unless the task explicitly uses them. A terminal win can fail a stricter task objective; a defensive-window success need not end the match.
Each model has one recorded attempt per task. These results do not estimate run-to-run variance.
Limitations
- One attempt per task. There are no estimates of run-to-run variance or uncertainty intervals.
- Tasks were developed using model feedback. Earlier cohorts and harder variants were selected partly from Opus and Sonnet behavior; this is not a held-out evaluation.
- Harnesses differ. Models use Claude Code, Codex, or mini-SWE-agent, with recorded harness-specific settings. The four mini-SWE runs have step_limit=12; 62/84 tasks terminate at that limit, and 33/84 are incomplete. Some scenarios require more moves than that budget supports under one-move-per-call play.