How WorldBuild Bench is scored.

A round is a set of shared game briefs and generated browser-game artifacts. Each row publishes the task, model, source run folder, generation time, cost context, code size, screenshots when available, and the playable game once it is published.

Automated harness probes can still exist in raw run artifacts, but they are treated as diagnostics only. They are not published as quality scores because this round showed they do not reliably match human-obvious failures in level layout, completion, controls, or world coherence.

The report still publishes operational data because it is useful for auditability and cost comparison. These fields do not decide which game is better:

Build status

Whether the generated project produced a browser-loadable build artifact. Broken builds can be dropped before Arena entry.

Generation time

Elapsed generation time is shown as process context, not as a quality score.

Cost

Generation cost per run.

Code size

Authored files and line counts are published for auditability, not ranked as quality.

Media

Screenshots and short clips are used for browsing; Arena scoring is based on opening the live games, with an explicit early exit for a clearly failed build.

Humans play two anonymous builds side by side and score one primary question:

“Which game would you rather keep playing?”

The centered slider allocates five preference points between the two games: center is 2.5/2.5, full left is 5/0, and full right is 0/5. The same lightweight slider format also captures game feel, world design, presentation, and human-observed completeness. A “both broken” submission is logged but excluded from the primary rating. If one build is clearly failed, the voter can score immediately; that ballot counts with an explicit early-exit flag rather than inventing play time. Model names reveal after submission whenever both games have at least five seconds of verified play.

The primary Arena Rating is fit with a Bradley-Terry model over the keep-playing preference signal and displayed on an Elo-like scale — anchor 1000, scale 400/ln(10): display = 1000 + (400 / ln 10) · ln(p_i / p_ref)— with a 95% confidence interval from bootstrap resampling. Category sliders produce separate diagnostic views rather than silently flattening everything into the headline. Ratings remainprovisional until the comparison graph is connected and every active model has at least 50 counted comparisons.

Anti-bias measures:

Hidden identity

Model names stay hidden while voting, then reveal after submission once both games have at least five seconds of verified play.

Side randomization

Which model lands on the left or right is randomized per pair.

Play review & early exit

The normal review is 20 seconds per side. A clearly failed build can be scored early without falsifying its measured play time; that exception is recorded explicitly.

Rate limits & dedup

Votes are IP-rate-limited server-side and deduplicated with a salted voter hash.

Exposure balancing

Pair selection prefers the least-voted pairs so no model farms exposure.

The public headline is the Arena Rating once enough human comparisons exist. It is computed from the primary keep-playing slider and shown with confidence intervals. The category sliders are reported separately so the site can say, for example, that one model wins presentation while another wins game feel or world design.

Honesty rule: until the round clears both publication conditions, no headline quality number is shown. The round page instead shows the generated builds and comparison progress.

Games enter the Arena only after they are published as Sandscape games and registered in the arena entry table for a specific round, task, and model. The Arena then serves two active entries from the same task with model identities hidden.

The vote endpoint records play time, side randomization, whether the voter exited the normal review early, the primary winner, and all five raw slider positions. Eligible votes feed the headline rating and the separately reported diagnostic category views.

Round JSON is downloadable from each round page. Legacy fields such as playability and world_coherence are intentionally null in public round data unless a future methodology reintroduces validated machine scoring.