AI Judge

Lab policy

Privacy

AI Judge is a personal benchmark lab run by MiMs — not a company and not a private workspace. This page says what we store, what is public, and what we do not do.

Effective 28 August 2026 · Last updated 28 August 2026 · Operator MiMs

Who runs this

The operator is MiMs, a person. There is no registered company behind this site. Contact is Discord only: MiMs (292675388180791297).

This is plain language for a hobby lab, not formal legal advice.

Age

You must be at least 13. Discord’s own rules also apply. If you are under 18, get a parent or guardian’s okay before you paste an OpenRouter key — that key can spend real money.

What we store when you sign in

Sign-in is Discord, identify scope only. We do not ask for your email and we do not store one.

  • Discord id, username, and avatar URL
  • A role on this site (user, moderator, or admin)
  • When the account was created
  • A signed-in session cookie so you stay logged in

Signing out clears the session cookie. Revoking the Discord app in Discord’s settings stops future logins.

Lab records are public

This is a shared instrument, not a private notebook. If you put something in a run, a playground chat, or a published pack, treat it as public.

  • Benchmark runs — listed on /runs. Snapshots, live workbenches, and JSON/CSV exports are open. Anyone who knows the run id can read answers and judge text. Only pause, resume, cancel, and retry stay locked to the person who launched the run.
  • Leaderboards, compare, and judges pages — built from every complete run on this host.
  • Playground chats — no login required. Transcripts and judgments are stored. Recent sessions are listed. Anyone with the session link can read them.
  • Published packs — listed on /bundles with your Discord name and avatar. Drafts stay visible only to you until you publish. Published packs cannot be edited or deleted.

Do not paste secrets, private keys, personal data about other people, or anything you would not put on a public noticeboard.

Your OpenRouter key

In production you paste your own key. It lives in this browser’s localStorage, is sent to this server only to call OpenRouter, and is held in process memory while a run or chat is live. It is not written to SQLite, not put in the session, and not shown back to you except the last four characters.

Discord never receives the key. This site does not bill you. OpenRouter bills the key. Read OpenRouter’s privacy policy.

What we send to model providers

When you launch a run, chat, test a key, refresh the catalog, or generate a pack, this server sends prompts and model output to OpenRouter using your key (or a server key in local/dev mode). That includes task text, candidate answers, judge prompts, and playground transcripts.

Those providers have their own logs and policies. We do not control them.

Visitor counts

Most page views send a tiny beacon to this site. We store a daily path count and a hashed visitor token — not your IP, user agent, referrer, or Discord id. Paths are stripped of query strings; ids in the path are collapsed. Rollups are kept about 200 days, then deleted. Staff can see those totals on /admin.

The visitor cookie is named aij_vid. If your browser sends Do Not Track or Global Privacy Control, the site does not send the beacon. Details are on Cookies.

Shared lab settings

Concurrency, trials, budget, and timeout defaults are one shared row for this host — not per account. Any signed-in user can change them. They are operator knobs, not personal preferences.

Other services

  • Discord — login and avatar images. Discord privacy.
  • OpenRouter — model calls and catalog. OpenRouter privacy.
  • The machine that hosts this app may keep ordinary server logs (that is outside this codebase).

We do not delete lab records

There is no “delete my account” or “forget me” button, and we do not promise to erase runs, chats, published packs, or scores if you ask. Those rows are the lab’s public record. The leaderboard only stays honest if completed work remains.

You can stop using the site, remove your OpenRouter key from this browser, and revoke Discord access. Visitor hashes age out on their own. Session cookies go away when you sign out or they expire.

Changes

If this page changes in a way that matters, the date at the top updates. Keep using the site after that and you are using the new version.