error during signers.sign_pdf

[inui-sdt2]

Hello All
I "sometimes" encountered the following error during signers.sign_pdf.
I am using different certificate for each signing. the code is same.
is this related pyhanko_certvalidator related issue? I wonder the root cause is certificate itself or validation?
please let me know if you have any ideas to resolve this.

$ python3 test_sign.py 
Traceback (most recent call last):
  File "test_sign.py", line 47, in <module>
    signers.sign_pdf(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pyhanko/sign/signers/functions.py", line 75, in sign_pdf
    return signer.sign_pdf(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pyhanko/sign/signers/pdf_signer.py", line 1221, in sign_pdf
    result = asyncio.run(
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 616, in run_until_complete
    return future.result()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pyhanko/sign/signers/pdf_signer.py", line 1273, in async_sign_pdf
    await signing_session.perform_presign_validation(pdf_out)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pyhanko/sign/signers/pdf_signer.py", line 1443, in perform_presign_validation
    async for ts_path in ts_paths:
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pyhanko/sign/timestamps/api.py", line 132, in validation_paths
    yield await validator.async_validate_usage(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pyhanko_certvalidator/__init__.py", line 283, in async_validate_usage
    await self._validate_path()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pyhanko_certvalidator/__init__.py", line 142, in _validate_path
    await async_validate_path(self._context, candidate_path, self._params)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pyhanko_certvalidator/validate.py", line 119, in async_validate_path
    return await _validate_path(validation_context, path, parameters=parameters)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pyhanko_certvalidator/validate.py", line 644, in _validate_path
    await _check_revocation(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pyhanko_certvalidator/validate.py", line 826, in _check_revocation
    await verify_ocsp_response(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pyhanko_certvalidator/validate.py", line 1423, in verify_ocsp_response
    ocsp_resp_issuer = await _ocsp_responder_issuer(
  File "/home/ubuntu/.local/lib/python3.8/site-packages/pyhanko_certvalidator/validate.py", line 1194, in _ocsp_responder_issuer
    if end_entity_name_override is None and responder_cert.sha256 != issuer.sha256:
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/x509.py", line 2858, in sha256
    self._sha256 = hashlib.sha256(self.dump()).digest()
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 4117, in dump
    self._set_contents(force=force)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 3671, in _set_contents
    child_dump = child.dump(force=force)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 4117, in dump
    self._set_contents(force=force)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 3671, in _set_contents
    child_dump = child.dump(force=force)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 4585, in dump
    self._set_contents(force=force)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 4457, in _set_contents
    contents.write(child.dump(force=force))
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 4117, in dump
    self._set_contents(force=force)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 3671, in _set_contents
    child_dump = child.dump(force=force)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 2883, in dump
    native = self.parsed.dump(force=force)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 4585, in dump
    self._set_contents(force=force)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 4457, in _set_contents
    contents.write(child.dump(force=force))
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 4117, in dump
    self._set_contents(force=force)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 3667, in _set_contents
    child_dump = self._lazy_child(index).dump(force=force)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 1721, in dump
    self.set(native)
  File "/home/ubuntu/.local/lib/python3.8/site-packages/asn1crypto/core.py", line 3100, in set
    part = int(part)
ValueError: invalid literal for int() with base 10: ''

[MatthiasValvekens]

This is very interesting. For some reason, either the OCSP responder's certificate or the issuer certificate can't be fully decoded (or encoded) by asn1crypto. This could be an encoding error in the affected certificate, a bug in asn1crypto (which isn't maintained by me), a bug in pyhanko_certvalidator or a combination of all of those.

Would it be possible to send the affected certificates (without the private key material, obviously) to pyhanko.samples@mvalvekens.be so I can investigate? If you're consistently getting this error with a certain (subject, issuer) pair then having those would probably suffice.

If you're not at liberty to disclose the certificates, I certainly understand, but then this issue is going to be tricky for me to diagnose accurately. In case you're familiar with X.509 and can read ASN.1, putting the certificates through an ASN.1 decoder could also be useful. :)

---

EDIT: While I don't maintain asn1crypto, I have sent some contributions upstream to them, and pyhanko_certvalidator patches in some of that functionality at runtime in anticipation of the next release. It's possible that that is related to the issue you're seeing. Can you also share the output of pip freeze or similar?

If you're not familiar with ASN.1, I can try to walk you through the process I would use to figure out whether the issue is related to pyhanko_certvalidator or not.

[inui-sdt2]

Dear @MatthiasValvekens
Thank you, I will send details.

[MatthiasValvekens]

Thanks for the files :)

I replied to your email and also put the full analysis as a separate answer below.

[MatthiasValvekens]

Alright, with some staring at the files that were sent to me (thanks, @inui-sdt2!), I figured out the root cause(s) of the problem.

TL;DR: There's no problem with your certificates. It's FreeTSA's OCSP responder that is broken. I'll look into hardening pyhanko-certvalidator to treat cases like this as it would any other OCSP fetching error. In the meantime, switching to a different TSA should suffice.

Here's an alternative option: http://timestamp.entrust.net/TSS/RFC3161sha2TS, operated by Entrust. Unlike FreeTSA, this one is on the AATL, so it can even be used in (most) production workflows.

If you want the full explanation, read on.

---

There are actually two different phenomena at play. First, the OCSP responder operated by FreeTSA indeed includes a malformed certificate in its responses (see attachment). The certificate policies extension on the certificate "identifies" its certification policy using an OID of length zero. This is not allowed in ASN.1, and leads to a parse error in asn1crypto.

Now, of course this does not explain why this error only crops up occasionally, and not on every single signature with a timestamp... The reason for that is actually that FreeTSA's OCSP responder is offline for the vast majority of the time! Since pyhanko-certvalidator is set up to verify revocation status by querying OCSP first, it notices that the OCSP responder is down and then falls back to downloading a CRL. The CRL fetch succeeds, and validation proceeds as usual. In the rare case that the OCSP request from pyhanko-certvalidator does get a response, FreeTSA's OCSP service returns the corrupt certificate I linked above, and validation fails with a parse error.

But it gets better: after a couple tries, I noticed that I was usually able to get in one OCSP request right at the hour mark. I suspect that FreeTSA's OCSP responder is scheduled to restart at the top of every hour, and that it simply crashes after the first request, which causes it to go down again until another hour goes by.

All things considered, this was a pretty interesting bug hunt!

[inui-sdt2]

Oh,,,What an interesting thing was going on backend! I couldn't find the reason on my own. thank you!