Issues with PAdES generation using remove signature

[hyzyla]

I'm trying to integrate mobile-id.vn with pyHanko, but for me, it doesn't work – generated PAdES is not valid. mobile-id.vn has API similar to CSC API:

Trusted Hub Service Provider Appliance QSCD - RESTful API Specification - V3.11.230418 (2).pdf

I created reproducible example and here how I'm doing signing:

    from rssp import load_rssp

    options = load_rssp()
    chain = options["chain"]
    signer = options["signer"]

    with open("input.pdf", "rb") as inf:
        w = IncrementalPdfFileWriter(inf)
        validation_context = ValidationContext(
            allow_fetching=False,
            trust_roots=[chain[-1]],
            other_certs=chain,
            algorithm_usage_policy=AcceptAllAlgorithms(),
        )

        meta = signers.PdfSignatureMetadata(
            field_name="MobileIdVnExample",
            subfilter=SigSeedSubFilter.PADES,
            validation_context=validation_context,
            embed_validation_info=False,
            use_pades_lta=False,
        )
        with open("output.pdf", "wb") as outf:
            await signers.async_sign_pdf(
                w,
                meta,
                signer=signer,
                output=outf,
            )

And here is how I sign using API:

class RSSPClient:
    ....

    def signHash(self, data: bytes) -> bytes:
        assert self.sad is not None
        assert self.access_token is not None

        doc_hash = base64.b64encode(data).decode()

        response = httpx.post(
            "https://rssp.mobile-id.vn/rssp/v2/signatures/signHash",
            json={
                "credentialID": "52fa1877-8163-47ca-913e-3d90afa3b584",
                "documentDigests": {
                    "hashes": [doc_hash],
                    "hashAlgorithmOID": "2.16.840.1.101.3.4.2.1",  # SHA256
                },
                "operationMode": "S",
                "signAlgo": "1.2.840.113549.1.1.1",  # RSA
                "SAD": self.sad,
                "signAlgoParams": "BgkqhkiG9w0BAQo=",
                "profile": "rssp-119.432-v2.0",
                "lang": "EN",
            },
            headers={
                "Authorization": "Bearer " + self.access_token,
            },
        )
        data = response.json()
        print("SignHash response:")
        pprint(data)
        signature_str = data["signatures"][0]
        signature = base64.b64decode(signature_str)

        return signature


class RSSPSigner(Signer):
    def __init__(
        self,
        signing_cert: x509.Certificate,
        signature_mechanism: algos.SignedDigestAlgorithm,
        client: RSSPClient,
        other_certs=(),
    ):
        cr = SimpleCertificateStore()
        cr.register_multiple(other_certs)

        self.client = client

        super().__init__(
            signature_mechanism=signature_mechanism,
            signing_cert=signing_cert,
            cert_registry=cr,
        )

    async def async_sign_raw(
        self,
        data: bytes,
        digest_algorithm: str,
        dry_run=False,
    ) -> bytes:
        print("============ SIGNING ============")
        print("Digest algorithm:", digest_algorithm)
        print("Data:", data, len(data))
        doc_hash = hashlib.sha256(data).digest()
        print("Doc hash:", doc_hash)
        return self.client.signHash(doc_hash)


def load_rssp():
    client = RSSPClient()
    client.login()
    client.download_certificates()
    client.authorize()

    cert: x509.Certificate = load_cert_from_pemder("./certificates/cert_0.der")
    chain = (
        load_cert_from_pemder("./certificates/cert_1.der"),
        load_cert_from_pemder("./certificates/cert_2.der"),
    )

    signer = RSSPSigner(
        signing_cert=cert,
        other_certs=chain,
        signature_mechanism=algos.SignedDigestAlgorithm({"algorithm": "sha256_rsa"}),
        client=client,
    )
    return {
        "cert": cert,
        "chain": chain,
        "signer": signer,
    }

Could you help me to understand why it doesn't work? Thank you in advance

[MatthiasValvekens]

Given that others (including myself) presumably don't have access to the API you're talking about and thus cannot run your code, we're going to need a little more info than that. ;)

What is the result you get, and why is it unexpected? If possible, it would help if you could upload a sample output document.

[hyzyla]

Sorry that I haven't mentioned earlier, that I've already included reproducible API keys in my example (demo account) to this API. It works in GitHub Codespace for me without any changes. It should work just by typing python main.py

Here is my output:
output.pdf

[hyzyla]

When I was trying to debug, I noticed that signature API is called twice and both time data to sign is equal to 274 bytes, but on verification data was fed to digest function is way longer, and I clearly see that it is PDF data.

[MatthiasValvekens]

I noticed that signature API is called twice and both time data to sign is equal to 274 bytes

Ah, that could be related. Your async_sign_raw implementation does not use the dry_run flag, which is used by pyHanko to perform size estimations before generating the actual signature container. With remote signers, you probably shouldn't make actual calls to the API when dry_run=True, but just return a payload of (approximately) the correct size. Is your SAD pinned to some piece of data that could change between these calls?

verification data was fed to digest function is way longer, and I clearly see that it is PDF data.

I suspect that you're looking at two very different stages of the process: the data that you send to the remote API to get signed is not actually the hash of the document. Rather, you're signing a collection of CMS attributes that will go into the signature container, and the document hash is one of those. Of course, at validation time, it's essential to (among many other things) verify both the signature and whether or not the message digest attribute matches the document hash. So this is by itself not unexpected.

I'll try to find time to analyse your file in more detail, but I can't make any promises as to when (I'm pretty swamped right now, unfortunately).