claude-opus-5 × judging

{ "ranking": ["Answer B", "Answer A"], "answer_a": { "scores": { "security_correctness": 0, "completeness": 1, "clarity": 7 }, "critical_issues": [ "Recommends storing plaintext passwords — a critical, well-known vulnerability and a violation of basic standards (e.g., OWASP ASVS, NIST SP 800-63B).", "False premise that database encryption prevents attackers from reading data: encryption at rest does not protect against SQL injection, compromised application credentials, insider access, backup leaks, or a live server with keys loaded.", "Endorses password recovery (returning the original password), which requires reversible storage and is unsafe; reset flows are the correct pattern.", "Ignores credential stuffing / password reuse harm: a plaintext breach compromises users' accounts on other services.", "No mention of salting, slow key-derivation/password-hashing functions, per-user salts, or peppering.", "Misleading performance claim: password hashing cost is deliberate and negligible at login volumes typical of early-stage SaaS; 'faster login' is not a valid tradeoff.", "Omits key management entirely — encryption security depends on where the key lives, which is often the same host.", "Likely creates compliance/legal exposure (e.g., GDPR 'appropriate technical measures', SOC 2, PCI-adjacent expectations)." ], "partial_strengths": [ "Correct that database encryption at rest does provide some protection, specifically against stolen disks, lost backups, or discarded hardware.", "Technically correct that skipping a slow hash reduces CPU work per login (though this is an intentional cost, not a benefit to pursue).", "Clearly and directly written, and it does answer the question asked rather than dodging." ] }, "answer_b": { "scores": { "security_correctness": 10, "completeness": 7, "clarity": 9 }, "critical_issues": [ "No critical security errors." ], "partial_strengths": [ "Correct core recommendation: slow, salted, purpose-built password hashing (Argon2id, bcrypt, scrypt).", "Correctly frames encryption at rest as defense in depth rather than a substitute.", "Correctly replaces password recovery with password reset.", "Minor gaps only: does not give parameter guidance (Argon2id memory/iterations, bcrypt cost ~12+), bcrypt's 72-byte input limit, use of a vetted library or managed identity provider rather than hand-rolled code, constant-time comparison, rehashing on login when parameters change, rate limiting and MFA, breached-password screening, and avoiding logging of credentials." ] }, "ideal_answer": "No. Never store recoverable passwords, even in an encrypted database; encryption at rest only protects stolen disks or backups, not SQL injection, leaked app credentials, or insiders holding the key. Store a per-user salted hash from a purpose-built slow function — Argon2id (preferred), scrypt, or bcrypt (cost 12+) — via a vetted library, and rehash when parameters change. Better still, delegate authentication to a managed identity provider. Add login rate limiting, MFA, breached-password screening, constant-time verification, and no credential logging. Offer password reset by emailed single-use expiring token, never password recovery." }