Read this first
Swapping RSA and ECDH for ML-KEM and ML-DSA is the easy part. The unglamorous, load-bearing question is whether the implementation you chose actually follows the standard. "It is an audited library" is not the same as "it conforms to FIPS 203." Sieve is a zero-dependency conformance battery that drives any implementation — in any language — through a tiny stdin/stdout protocol and reports where it deviates. This post shows how to run it, and a genuine FIPS 203 deviation it found in a widely-used library (one the maintainers had already fixed upstream — more on that below).
Audits look for exploitable bugs. Conformance is a different axis entirely: does your ML-KEM encapsulation reject a malformed key the way FIPS 203 §7.2 says it must? Are your ML-KEM-768 public keys exactly 1184 bytes? Does decapsulation perform implicit rejection instead of throwing on a bad ciphertext? A library can be perfectly safe for honest use and still deviate from the letter of the standard — and if you are the one signing the compliance attestation, that gap is yours.
What conformance testing actually means here
Sieve tests other people's implementations. It deliberately does not implement ML-KEM / ML-DSA / SLH-DSA itself, and — this matters — it never fabricates Known-Answer-Test values. A conformance tool that invents its own "expected" ciphertexts is only testing its own math. Sieve's authority comes from two honest sources.
- Checks that need no secret vectors — self-consistency, structural, and negative properties every conforming implementation must satisfy. Encapsulate then decapsulate: you must recover the same shared secret. Flip a byte of the ciphertext: ML-KEM must implicitly reject (return a pseudorandom secret), not throw. Feed a key one coefficient out of range: §7.2 says reject it.
- A loader for the official NIST ACVP / KAT vectors you download — exact-byte checks whose expected values come from your NIST files, never from Sieve.
The only cryptographic constants baked into Sieve are the public, standardized parameter sizes — an ML-KEM-768 public key is 1184 bytes; an SLH-DSA-SHA2-128f signature is 17088 bytes (FIPS 205, Table 2). Those are not secrets, and they are a surprisingly effective tripwire.
One small protocol, any language
Sieve spawns your implementation as a child process — the system under test — and talks to it over newline-delimited JSON on stdin and stdout, with byte fields base64-encoded: one request per line in, one response per line out. Because the contract is just JSON over pipes, your implementation can be in any language — Rust, Go, Python, C, a WASM build, or a cloud KMS behind a shim.
You write a thin adapter — about thirty lines — that maps the protocol's keygen, encapsulate, decapsulate, sign and verify operations onto your library, then point Sieve at it and let it drive. It reports each category — correctness, determinism, implicit-rejection, robustness, sizes — and an overall PASS or FAIL. The known-answer-test line reads SKIP until you supply official vectors; Sieve ships none and will not pretend otherwise.
The proof: a real (since-fixed) deviation in a widely-used library
This is the part that convinced us the exercise was worth shipping. We ran Sieve against an older release of @noble/post-quantum — a widely-used, pure-JavaScript implementation of all three standards from the well-regarded noble family (whose sibling libraries noble-curves/ciphers/hashes are independently audited; noble-post-quantum itself is self-audited as of 0.6.1). ML-DSA-65 and SLH-DSA passed their self-consistency batteries cleanly. ML-KEM-768 passed correctness, determinism, implicit-rejection and robustness — and then flagged one thing.
The finding — FIPS 203 §7.2 encapsulation-key check
Sieve feeds encapsulation a correctly-sized key with a single coefficient forced out of range (at or above q = 3329). A §7.2-conformant implementation must reject that input — the modulus check. In @noble/post-quantum 0.5.x, ML-KEM accepted it and returned a success result instead. Credit where due: the maintainers had already added the modulus-reduction fix in 0.6.0 (March 2026), so current versions reject it correctly — our run simply hit an older pinned release. It was never exploitable — the §7.2 check is defense-in-depth, commonly omitted, and does not affect honest interoperability. But it was a genuine deviation from the letter of FIPS 203, on widely-used code — exactly the class of gap a conformance battery exists to surface, and precisely the kind that gets fixed once it is.
That is the gap between "safe" and "conformant," made concrete — the kind of thing that matters when you map an implementation to a compliance requirement like Common Criteria, a FIPS 140-3 module boundary, or an ISO/IEC 27001 A.8.24 control.
Wire it into CI
Conformance is not a one-time check — implementations get updated, parameter sets get swapped, someone bumps a dependency. Pin it in CI: run Sieve with JSON output and fail the build unless the overall verdict is PASS. Sieve exits non-zero on a failure, so it gates the build like any other test, with zero runtime dependencies — nothing new to vet in your supply chain.
Where this fits
Post-quantum migration is two problems, not one.
- Find the classical crypto that has to go — that is qScan and the GitHub Action: the harvest-now-decrypt-later surface, across your languages, config, and infrastructure-as-code.
- Prove what you replaced it with is correct — that is Sieve.
Neither replaces the other, and neither is a full PQC library — you still bring liboqs, @noble/post-quantum, or a vetted native implementation. Sieve is the ruler you hold up against it. If you are migrating to ML-KEM / ML-DSA / SLH-DSA, spend the five minutes to write the adapter and run the battery before you sign anything that says "post-quantum ready." You may be surprised what a standard actually asks for.
A conformance tool that fabricates its own expected values is only testing its own math. Sieve's power is that it never does.
One boundary to keep clear: conformance testing is not the same as formal validation. NIST's Cryptographic Algorithm Validation Program (CAVP), driven by ACVP vectors, is what earns a certificate; a FIPS 140-3 module goes further still through the CMVP. Sieve is a fast pre-screen you run yourself — it catches deviations early and cheaply, but it is not a substitute for CAVP/CMVP where a certificate is required.
References
- NIST standards: FIPS 203 / ML-KEM (the §7.2/§7.3 input checks and Table 3 sizes cited above), FIPS 204 / ML-DSA, FIPS 205 / SLH-DSA.
- NIST: SP 800-227 — Recommendations for Key-Encapsulation Mechanisms (where key-check assurance "through other means" lives), and the ACVP project + ML-KEM vector spec.
- Adversarial vectors: C2SP / CCTV ML-KEM test vectors — community edge-case vectors, including the out-of-range coefficient trap the finding hit.
- The library in the example: @noble/post-quantum (its README documents the modulus-reduction fix in 0.6.0).