Polymarket benchmark · live · cumulative since May 6, 2026
We resolve prediction-market questions whose answer lives at a public URL. Polymarket is our live testbed. Every market runs through a six-step pipeline of deliberate filtering decisions, public sources, and verified gates.
Every market ending in the cumulative window.
Chainlink and Pyth resolve via on-chain price feeds. The chain already has the answer.
Remaining markets with a usable source path. This is the Intelligent Oracle’s universe — the 100% the pie chart breaks down.
The Intelligent Oracle fetches the host Polymarket named.
Named host blocks validators. We route to a verified alternate.
Paywall, login wall, captcha, or pure-consensus markets. They wait for the right resolution mode.
Polymarket today resolves prediction markets through three on-chain oracles: UMA, Chainlink, and Pyth. This benchmark walks every polled market through a five-step funnel to determine whether the Intelligent Oracle can independently resolve it from a source URL. On every market we can resolve, our outcome matches Polymarket’s — 100% agreement on the resolvable set.
Polymarket already routes price-feed and asset markets to deterministic on-chain oracles — Chainlink and Pyth. These are the right tool for that job, so the Intelligent Oracle steps aside. Everything else continues into the funnel.
Entering this phase
Is this a market where the Intelligent Oracle has a role?
On-chain feed — out of scope
Already resolved deterministically by Chainlink or Pyth.
No URL anywhere
No source URL appears in resolution criteria or description.
Some of these markets reference facts we could resolve without a URL — e.g., via off-chain agentic search or by inferring the canonical source from market context. Tracked as a research direction.
Entering this phase
Does the market reference any source URL at all?
The Intelligent Oracle reads the real world through URLs. If the market’s resolution criteria and description never reference one, there is nothing to read — we hold those for human review.
The cleanest case: the criteria links to a deep page that already contains the answer. The Intelligent Oracle fetches it and resolves — no rewriting, no search, no substitution.
Entering this phase
Does the exact URL in the Polymarket criteria resolve the market directly?
Resolved by exact source
Direct fetch of the criteria URL produces the answer.
Resolved by deep search
A deeper or more specific page inside the same named source family.
Entering this phase
Can the Intelligent Oracle find the right page inside the same source family?
Often the criteria points to a homepage or section root. An agent crawls the same source, finds the specific page that contains the resolution fact, and reads it.
When the named source is unusable, the Intelligent Oracle reaches for a verified alternative — Liquipedia, bo3.gg, Flashscore, ESPN Cricket, Wikipedia — and resolves from there. The remaining markets are held for human review.
Entering this phase
Can the same fact be resolved from a verified alternative source?
Resolved by alternative
Verified alternative source such as Liquipedia, bo3.gg, Flashscore, ESPN Cricket, or Wikipedia.
Today these 1,025 markets fall outside the Intelligent Oracle’s automated reach. They could be routed to UMA, settled by a Polymarket admin, or queued for a human reviewer. None of them have been resolved incorrectly — we simply do not assert an outcome.
Held for human review
What happens to the markets the Intelligent Oracle cannot resolve today?
Most of this set is solvable with work we have not built yet. Each item below is an active research direction.
Each row is one day’s contribution to the cumulative dataset. Cells deep-link into the explorer.
| Date | Markets | Chainlink | Pyth | Direct source | Alternative source | Held / unresolvable |
|---|---|---|---|---|---|---|
| 2026-05-06 | 4,132 | 2,730 (66.1%) | 42 (1.0%) | 912 (22.1%) | 366 (8.9%) | 82 (2.0%) |
| 2026-05-07 | 3,713 | 2,730 (73.5%) | 42 (1.1%) | 673 (18.1%) | 170 (4.6%) | 98 (2.6%) |
| 2026-05-08 | 9,503 | 2,730 (28.7%) | 322 (3.4%) | 4,303 (45.3%) | 1,805 (19.0%) | 343 (3.6%) |
| 2026-05-09 | 11,710 | 2,730 (23.3%) | 0 | 4,897 (41.8%) | 3,994 (34.1%) | 89 (0.8%) |
| 2026-05-10 | 11,125 | 2,730 (24.5%) | 0 | 4,100 (36.9%) | 4,220 (37.9%) | 75 (0.7%) |
| 2026-05-11 | 7,851 | 2,730 (34.8%) | 41 (0.5%) | 2,999 (38.2%) | 1,908 (24.3%) | 173 (2.2%) |
| 2026-05-12 | 7,924 | 2,730 (34.5%) | 42 (0.5%) | 2,290 (28.9%) | 2,789 (35.2%) | 73 (0.9%) |
| 2026-05-13 | 6,310 | 2,275 (36.1%) | 42 (0.7%) | 2,200 (34.9%) | 1,715 (27.2%) | 78 (1.2%) |
| Total | 62,268 | 21,385 (34.3%) | 531 (0.9%) | 22,374 (35.9%) | 16,967 (27.2%) | 1,011 (1.6%) |
Every market, every source host, every cut. One filter bar drives every section.
Explore →The pipeline, the gates, the open questions. Reference depth for the curious.
Read →One row per market: ID, slug, question, named source, verified source, status, bucket, settlement winner.