Also mentioned
ML-KEMML-KEMModule-Lattice-based Key Encapsulation MechanismModule-Lattice-Based Key-Encapsulation Mechanism, the NIST-standardized post-quantum KEM derived from CRYSTALS-Kyber and specified in FIPS 203.Read the full entry, ML-DSAML-DSAModule-Lattice-based Digital Signature AlgorithmModule-Lattice-Based Digital Signature Algorithm, the NIST-standardized post-quantum signature scheme derived from CRYSTALS-Dilithium and specified in FIPS 204.Read the full entry, SLH-DSASLH-DSAStateless Hash-based Digital Signature AlgorithmStateless Hash-Based Digital Signature Algorithm, the NIST-standardized signature scheme derived from SPHINCS+ and specified in FIPS 205.Read the full entry, FIPSFIPSFederal Information Processing StandardFederal Information Processing Standards, publicly announced standards developed by NIST for use in U.S. government computer systems, including cryptographic algorithms and modules.Read the full entry, NISTNISTNational Institute of Standards and TechnologyThe U.S. National Institute of Standards and Technology, the agency that develops and publishes cryptographic standards, including the FIPS series and post-quantum algorithms.Read the full entry, ACVPACVPAutomated Cryptographic Validation ProtocolThe Automated Cryptographic Validation Protocol, a NIST system for automatically testing that cryptographic algorithm implementations conform to their specifications.Read the full entry, KATKATknown answer testA test that feeds an implementation a fixed input with a published correct output and checks that it produces exactly that.Read the full entry are defined in the glossary.
Read this first
Three separate questions hide behind "is our post-quantum crypto correct?" Does it pass NIST's known-answer test (KAT) vectors? Does it conform to the letter of FIPS 203/204/205, including the reject paths? Is it secure in your deployment? These are three distinct axes, and passing the first proves neither of the others. KATs only exercise honest, well-formed inputs; conformance and security both live in the behaviour KATs never trigger.
When teams adopt ML-KEM or ML-DSA, the natural correctness check is NIST's Automated Cryptographic Validation Protocol (ACVP) test vectors: feed the fixed inputs, compare the outputs, all green, done. That check is necessary but it is not conformance, and conformance is not security. Conflating the three is how nonconformant implementations end up in production carrying a clean bill of health.
What the KATs miss
Known-answer test vectors pin the happy path. They confirm that for a given seed and message the implementation produces exactly the byte string the standard says it should. That catches transcription errors, endianness bugs, and wrong parameter sets. What it cannot catch is everything the standard requires an implementation to do with input it should refuse, because a KAT, by construction, only ever supplies input a conformant implementation accepts.
FIPS 203 is the clearest example. The algorithm is only half the standard; the input checks are normative:
- §7.2, encapsulation-key modulus check. Every coefficient of the encapsulation key must be reduced mod q = 3329. A key with an out-of-range coefficient must be rejected. No honest KAT vector contains such a key, so a build that skips the check passes the vectors regardless.
- §7.3, ciphertext type check. Decapsulation must verify the ciphertext is the correct length for the parameter set before processing it.
- Implicit rejection. On an invalid ciphertext, decapsulation does not error: it returns a deterministic pseudo-random shared secret. That is only safe if the reject path is indistinguishable, by timing, from the accept path. A branch that shortcuts on failure is nonconformant and creates an oracle.
ML-DSA has the same shape: verification must reject malformed public keys and signatures rather than crash or silently accept, and it must apply domain separation and the context string correctly, behaviour the standard mandates but the honest vectors never stress. SLH-DSA adds its own encoding and length constraints. In every case the load-bearing behaviour is on the reject path, which is exactly the path KATs cannot reach.
Conformance versus audit
Conformance and audit answer different questions and catch different failures. Conformance asks: does this implementation follow the specification, including every input check and reject path? Audit asks: is it exploitable in context, meaning side channels, RNG quality, key management, memory handling? You want both. A conformant implementation can still be insecure (a perfect §7.2 check leaks nothing, but a data-dependent timing branch elsewhere does); an audited implementation can still be nonconformant (secure in your threat model yet skipping a check that breaks interoperability with a peer that enforces it). Neither subsumes the other.
"It's an audited, popular library" is not evidence of conformance
Adoption and an audit report tell you a project is scrutinized; they do not tell you it conforms to FIPS 203. Real, widely deployed implementations have passed every ACVP vector while omitting the §7.2 modulus check or leaking the implicit-rejection path by timing. Popularity selects for the happy path being right, which is precisely the part KATs already cover and conformance does not depend on.
How conformance testing works
The method is to drive the implementation, in any language, through a small stdin/stdout protocol and feed it adversarial vectors that honest KATs never include: out-of-range coefficients, malformed keys, wrong-length ciphertexts, and boundary encodings, such as those the C2SP/CCTV project publishes for ML-KEM. You then assert the standard-mandated response: rejection where the spec requires it, an indistinguishable deterministic secret on implicit rejection, correct sizes throughout.
- Wrap, don't rewrite. A thin harness exercises the real code path you ship, not a reimplementation, so the test reflects production behaviour.
- Adversarial vectors, not just KATs. The value is in the inputs a conformant implementation must refuse; sourcing them from a maintained corpus keeps the suite honest.
- Re-run every release. Conformance is not a one-time certificate: a dependency bump or optimization can quietly drop a check. Gate releases on it, and emit findings in a machine-readable format like SARIF so they land in your existing review flow.
- Produce a signed report. The output is a dated, signed conformance statement you can show auditors, customers, and regulators.
Testing the reject paths the vectors never reach
We run all three axes so none of them hides behind another. We conformance-test the implementation you ship against the FIPS 203/204/205 input checks and reject paths the KATs miss (certification), assess whether it is secure in your deployment (audit), and sequence the change into your rollout without breaking production (migration). The deliverable is a signed conformance report backed by adversarial vectors and re-run on every release. That is evidence, not a vendor's assurance.