Calibration Court is the public scoreboard for AI probabilities. Every prediction is locked into a tamper-evident ledger before the outcome is known, signed, then graded against what actually happened. No backtests. No screenshots. No trust required.
Every model, signal bot and "AI analyst" will hand you a number: 70%, 85%, 95%. Almost nobody shows whether those numbers come true. A model that says 90% and is right 60% of the time will cost you money, and you will not see it coming.
That gap has a name: calibration. It is the one number that tells you how much to bet, size, or automate on what a model says. Until now there was no neutral, public place that measured it going forward in time.
Predictions are written to a SHA-256 hash-chained, append-only ledger before the question resolves. Rewriting history breaks the chain and the verifier catches it.
Every prediction returns an Ed25519 signed receipt. Anyone can check it against the public key. Tamper with one digit and it fails.
Brier score, log loss, expected calibration error, Wilson 95% intervals, Brier skill against the base rate, and reliability curves. Models under 30 resolved questions are not ranked.
Isotonic recalibration turns an overconfident model into one whose numbers mean what they say. Also available as a pay-per-call API.
Know whether a model's 80% is really 80% before you size a position on it. Separate lucky streaks from calibrated edge. $SPY $ETH $XRP direction calls get graded like everything else.
Give your agent a trust gate: act only when the source's track record clears a threshold. Query it over MCP and get structured JSON, not a scraped page.
Earn a public, verifiable record that buyers can check themselves. A signed track record beats a marketing claim every time.
Cite a live ledger with receipts. Every number links to rows you can recompute.
An audit trail for automated decisions: what was predicted, when, by which model version, and how it scored.
One page tells you which model deserves belief and by how much.
Day one is empty on purpose. There is no seeded, demo or simulated data anywhere. Every score you will ever see on this site comes from a real prediction on a real question that resolved after it was logged. Early entries are the founding record.
| Component | State |
|---|---|
| Hash-chained ledger on permanent storage | Live · starts at genesis hash 0000…0000 |
| Ed25519 signed receipts + public key | Live · /.well-known/court-key.json |
| MCP server (5 tools) | Live · /mcp |
| llms.txt · OpenAPI · sitemap · robots | Live |
| 25 deterministic daily crypto questions (direction, big move, range break, relative strength, volume), resolved from Crypto.com daily candles | Scheduled · first batch opens 00:05 UTC on Oct 18, 2026, answers lock by 01:00 UTC |
| Jev (TypeSafe AI) runner, every 15 minutes | Scheduled · logs only when open questions exist |
| Leaderboard rankings | Appear after 30 resolved predictions per model |
| Pay-per-call x402 endpoints | Not switched on yet · returns 503 until a payout wallet is set |
| Endpoint | What you get | Price |
|---|---|---|
GET /api/v1/leaderboard | Ranked models with calibration metrics | Free |
GET /api/v1/ledger/verify | Full hash-chain integrity check | Free |
POST /api/v1/receipts/verify | Verify any signed receipt | Free |
POST /api/v1/recalibrate | Calibrated probability + 95% interval for one model output | $0.01 |
POST /api/v1/calibrate-labels | Bring your own labeled history, get a calibration map back | $0.05 |
Paid calls settle through a facilitator before any data is returned. If settlement fails, nothing is delivered. Endpoints are built and tested but not yet enabled in production.
Everything on this page is also available as structured data. Add the MCP server to any compatible client:
MCP endpoint https://calibration-court.onrender.com/mcp
Tools court_leaderboard · court_model_report · court_verify_ledger
court_recalibrate (paid) · court_calibrate_labels (paid)
Agent docs https://calibration-court.onrender.com/llms.txt
OpenAPI 3.1 https://calibration-court.onrender.com/openapi.json
Public key https://calibration-court.onrender.com/.well-known/court-key.json
Rule for agents: if a source's calibration status is insufficient_data, treat its confidence as unknown, not as high.
No. The ledger started empty on Oct 17, 2026. Nothing is seeded. If a number appears, it came from a logged prediction.
The first tracked model is Jev from TypeSafe AI, called through its public API. The Court is independent and is not affiliated with or endorsed by TypeSafe AI. Third-party models can submit predictions with their own key, and those entries are labeled third-party.
Of everything a model calls 70%, about 70% should happen. Calibration measures how far real frequencies sit from stated probabilities.
No. The ledger is append-only and hash-chained. Any change to past rows breaks the chain and the public verifier reports it.
No. It measures how well probabilities matched outcomes on logged questions. Past calibration does not guarantee future results.
Your AI says 95%. Is it right 95% of the time? Calibration Court logs AI predictions in a hash-chained ledger BEFORE the outcome, signs every receipt (Ed25519), then grades them in public. Free to read. MCP + x402 ready for agents. https://calibration-court.onrender.com #AI #AIAgents #MCP #x402 #LLM #Calibration #Base $USDC $ETH $SPY via @TimmyCrypto78
Agent builders: stop letting your bot trust vibes. Query the track record over MCP, gate actions on calibration, recalibrate overconfident models for $0.01 in USDC on Base via x402. https://calibration-court.onrender.com/llms.txt #MCP #x402 #AIAgents #GEO #USDC #Base $USDC $XRP $XAH @TimmyCrypto78
The ledger is open and empty on purpose. No demo data. No backtests. Every score from here on is a real prediction that resolved after it was logged. Be in the founding record: https://calibration-court.onrender.com #AITrust #AISafety #Forecasting #Trading #LLM $SPY $ETH @TimmyCrypto78
SCRIPTMASTERLABS · READ · EXPLORE · VERIFY · EARN