CLI: Trouble to verfiy signed pdfs with pfx certificate store #77

thomasgundlach · 2022-02-14T19:03:25Z

thomasgundlach
Feb 14, 2022

first: Thank so much for that creating this great project. I'm continue with Python because of pyHanko!

After verify successful other pdfs by python and cli i started today to sign the first pdf myself.
Signing seems to be successful, but i have problems to verify them.

Here is what i'm doing with pyHanko-0.12.0:

Download root and subs and create pkcs12 certificate store

gundlac@batch-ost:~/mytest2$ curl -o RapidSSL_TLS_DV_RSA_Mixed_SHA256_2020_CA-1.crt.pem https://cacerts.digicert.com/RapidSSLTLSDVRSAMixedSHA2562020CA-1.crt.pem
gundlac@batch-ost:~/mytest2$ curl -o DigiCertGlobalRootCA.crt.pem https://cacerts.digicert.com/DigiCertGlobalRootCA.crt.pem
gundlac@batch-ost:~/mytest2$ cat star.domain.crt RapidSSL_TLS_DV_RSA_Mixed_SHA256_2020_CA-1.crt.pem DigiCertGlobalRootCA.crt.pem > new_cert.pem
gundlac@batch-ost:~/mytest2$ openssl pkcs12 -export -in new_cert.pem -inkey private_encrypted_key -out wc21.domain.p12
Enter Export Password:
Verifying - Enter Export Password:

Check that all certificates in the chain of trust and supplied KeyUsages

gundlac@batch-ost:~/mytest2$ keytool -J-Duser.language=en -list -v -storetype pkcs12 -keystore wc.domain.p12
Enter keystore password:

Keystore type: PKCS12
Keystore provider: SunJSSE

Your keystore contains 1 entry

Alias name: jetty
Creation date: Feb 14, 2022
Entry type: PrivateKeyEntry
Certificate chain length: 3
...
Certificate[1]:
Owner: CN=*.hanse-merkur.de
Issuer: CN=RapidSSL TLS DV RSA Mixed SHA256 2020 CA-1, O=DigiCert Inc, C=US
Serial number: 1cfdb574c3e0a43e8d6d764ccb7b81a
Valid from: Thu Jan 28 01:00:00 CET 2021 until: Tue Mar 01 00:59:59 CET 2022
...
#3: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Key_CertSign
  Crl_Sign
]
...
Certificate[2]:
Owner: CN=RapidSSL TLS DV RSA Mixed SHA256 2020 CA-1, O=DigiCert Inc, C=US
Issuer: CN=DigiCert Global Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Serial number: 7983603ade39908219ca00c27bc8a6c
Valid from: Thu Jul 16 14:25:27 CEST 2020 until: Thu Jun 01 01:59:59 CEST 2023
...
#7: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Key_CertSign
  Crl_Sign
]
...
Certificate[3]:
Owner: CN=DigiCert Global Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Issuer: CN=DigiCert Global Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US
Serial number: 83be056904246b1a1756ac95991c74a
Valid from: Fri Nov 10 01:00:00 CET 2006 until: Mon Nov 10 01:00:00 CET 2031
...
#8: ObjectId: 2.5.29.15 Criticality=true
KeyUsage [
  DigitalSignature
  Key_Encipherment
]
...

creating configuration for testing purposes to ignore missing non repudiation key extension because i cannot recreate the certifiate shortly

gundlac@batch-ost:~/mytest2$ vi pyhanko.yml
validation-contexts:
    setup-a:
        trust: star.domain.crt
        trust-replace: true
        other-certs: anchor.pem
        signer-key-usage: ["digital_signature"]

sign pdf seems ok

gundlac@batch-ost:~/mytest2$ pyhanko --verbose sign addsig --field Sig1 pkcs12 input.pdf output-pyhankoo.pdf wc.domain.p12
2022-02-14 17:29:29,740 - root - DEBUG - Running with --verbose
2022-02-14 17:29:29,740 - root - DEBUG - There was no configuration to parse.
PKCS#12 passphrase:
2022-02-14 17:29:38,698 - asyncio - DEBUG - Using selector: EpollSelector

anyhow validation failed unfortunately because of non repudiation and failed Chain of trust validation

gundlac@batch-ost:~/mytest2$ pyhanko sign validate --pretty-print output-pyhankoo.pdf
2022-02-14 17:30:05,397 - pyhanko.sign.validation.status - WARNING - The active key usage policy requires at least one of the key usage extensions non repudiation to be present.
2022-02-14 17:30:05,397 - pyhanko.sign.validation.status - WARNING - Chain of trust validation for Common Name: *.domain failed.
=============
Field 1: Sig1
=============


Signer info
-----------
Certificate subject: "Common Name: *.domain"
Certificate SHA1 fingerprint: 4e2b630023cb46c780f7383b8eed3acdcbe6d226
Certificate SHA256 fingerprint: 4f61b67b01ad745fb395b14a447692c1300bd26446800cd1637dc0ab9cb16e49
Trust anchor: "No path to trust anchor found."
The signer's certificate is untrusted.


Integrity
---------
The signature is cryptographically sound.

The digest algorithm used was 'sha256'.
The signature mechanism used was 'sha256_rsa'.


Signing time
------------
Signing time as reported by signer: 2022-02-14T16:29:38+00:00


Modifications
-------------
The signature covers the entire file.


Bottom line
-----------
The signature is judged INVALID.


Error: Validation failed

checking that root certifiate will be trusted

gundlac@batch-ost:~/mytest2$ ls -altr /etc/ssl/certs/ | grep -i DigiCert_Global_Root_CA
lrwxrwxrwx 1 root root     62 Dez 22 16:55 DigiCert_Global_Root_CA.pem -> /usr/share/ca-certificates/mozilla/DigiCert_Global_Root_CA.crt
lrwxrwxrwx 1 root root     27 Dez 22 16:55 3513523f.0 -> DigiCert_Global_Root_CA.pem

create pem file with sub and root certifiate to explicit embed them

gundlac@batch-ost:~/mytest2$ cat RapidSSL_TLS_DV_RSA_Mixed_SHA256_2020_CA-1.crt.pem DigiCertGlobalRootCA.crt.pem > anchor.pem

resigning pdf seems good

gundlac@batch-ost:~/mytest2$ pyhanko --verbose sign addsig --field Sig1 pkcs12 --chain anchor.pem input.pdf output-pyhankoo.pdf wc.domain.p12
2022-02-14 17:37:44,620 - root - DEBUG - Running with --verbose
2022-02-14 17:37:44,620 - root - DEBUG - There was no configuration to parse.
PKCS#12 passphrase:
2022-02-14 17:37:49.483 - asyncio - DEBUG - Using selector: EpollSelector

validation faileds again

gundlac@batch-ost:~/mytest2$ pyhanko --verbose sign validate --pretty-print output-pyhankoo.pdf
2022-02-14 17:48:13,720 - root - DEBUG - Running with --verbose
2022-02-14 17:48:13,720 - root - DEBUG - There was no configuration to parse.
2022-02-14 17:48:13,845 - asyncio - DEBUG - Using selector: EpollSelector
2022-02-14 17:48:13,870 - pyhanko.sign.validation.status - WARNING - The active key usage policy requires at least one of the key usage extensions non repudiation to be present.
2022-02-14 17:48:13,870 - pyhanko.sign.validation.status - WARNING - Chain of trust validation for Common Name: *.domain failed.
=============
Field 1: Sig1
=============


Signer info
-----------
Certificate subject: "Common Name: *.domain"
Certificate SHA1 fingerprint: 4e2b630023cb46c780f7383b8eed3acdcbe6d226
Certificate SHA256 fingerprint: 4f61b67b01ad745fb395b14a447692c1300bd26446800cd1637dc0ab9cb16e49
Trust anchor: "No path to trust anchor found."
The signer's certificate is untrusted.


Integrity
---------
The signature is cryptographically sound.

The digest algorithm used was 'sha256'.
The signature mechanism used was 'sha256_rsa'.


Signing time
------------
Signing time as reported by signer: 2022-02-14T16:29:38+00:00


Modifications
-------------
The signature covers the entire file.


Bottom line
-----------
The signature is judged INVALID.


Error: Validation failed

The pkcs12 certificate store with the certifiate and the chain of trust accepted by google chrome and other browsers as valid.
Do you have any hint how i can debug the problem furher? Or will i doing something completely wrong? Is there maybe a way to extract the signature which pyhanko add to the pdf?

Best Regards,

Thomas

Answered by MatthiasValvekens

Feb 14, 2022

Hi Thomas,

Thanks for checking in, and for the detailed report! Reading your description, three things come to mind:

The default key policy also needs to be overruled when validating a signature involving a certificate without the nonRepudiation key usage bit set. In other words, you need to ensure that your custom validation context is also used when validating the signature (assuming you didn't set it as the default). In your case, that'd mean passing --validation-context setup-a to pyhanko sign validate.\
Similarly, pyHanko's default chain of trust (as imported through oscrypto) may or may not agree with your browser's or favourite PDF viewer's trust store. How that's best addressed d…

View full answer

MatthiasValvekens · 2022-02-14T19:58:35Z

MatthiasValvekens
Feb 14, 2022
Maintainer

Hi Thomas,

Thanks for checking in, and for the detailed report! Reading your description, three things come to mind:

The default key policy also needs to be overruled when validating a signature involving a certificate without the nonRepudiation key usage bit set. In other words, you need to ensure that your custom validation context is also used when validating the signature (assuming you didn't set it as the default). In your case, that'd mean passing --validation-context setup-a to pyhanko sign validate.\
Similarly, pyHanko's default chain of trust (as imported through oscrypto) may or may not agree with your browser's or favourite PDF viewer's trust store. How that's best addressed depends on the use case, but again, if you need to rely on a trust root that's not in the system trust already, you need to tell pyHanko about it (I can elaborate as to why that's the case, if you'd like to know). Passing the right validation context on the command line should do the trick.
I find it somewhat strange that you're using what appears to be a TLS client certificate to sign a PDF document. In most "real world" PKI environments, TLS client certificates and document signing certificates are totally separate things. I'd be surprised if a PDF viewer like Acrobat would even accept such a certificate (assuming default settings). Are you signing with some sort of test certificate? What's the use case here?

TL;DR: Passing --validation-context setup-a to the validate subcommand will (probably) resolve your immediate problem, but the fact that you're running into this might point to a deeper issue with your workflow.

Hope that helps.

0 replies

thomasgundlach · 2022-02-15T00:37:59Z

thomasgundlach
Feb 15, 2022
Author

Hej @MatthiasValvekens,

thank you very much. That was the problem. Now it works like a charm:

pyhanko --verbose sign addsig --field Sig1 pkcs12 input.pdf output-pyhankoo.pdf wc.domain.p12
pyhanko --config \
  <(echo "validation-contexts:"; 
    echo "    setup-a:";
    echo "        signer-key-usage: ["digital_signature"]";
    ) \
  --verbose sign validate --validation-context setup-a --pretty-print output-pyhankoo.pdf

You right. The certificate is not common for document signing. I just had nothing other ca signed certificate and was not sure what kind of certificate i need for request.
So i was just testing to make sure that the technical part works.

I had wrote a little script in python with pyhanko to validate signed pdfs. Now i need to test that script still working after system upgrades. For that i need an other script which creates signed test pdfs - to test the first script automatically.

For that i will request a common certifiate for document signing tomorrow:

openssl req -newkey rsa:4096 -sha256 -keyform PEM -keyout key.pem -out cert-request.csr \
    -subj '/C=DE/ST=State/L=City/O=Organisation/OU=Group/CN=pyhanko_Signaturtest/emailAddress=mail@address' \
	-addext "keyUsage = critical,nonRepudiation,digitalSignature" \
	-addext "extendedKeyUsage = critical,codeSigning"
openssl req -text -in cert-request.csr

Thank you so much from Hamburg and have a good night.

1 reply

MatthiasValvekens Feb 15, 2022
Maintainer

OK, great! As far as testing goes, there's another project of mine that you might be interested in: https://github.com/MatthiasValvekens/certomancer. It can be used to generate test certs based on YAML, but also to spin up OCSP responders and CRL repositories for testing (and it powers a very significant part of pyHanko's own test suite).

Anyway, if openssl works for you, then you don't need to switch for the sake of switching, but if you want to try out some more complex stuff: maybe give Certomancer a spin :)

thomasgundlach · 2022-02-15T11:02:14Z

thomasgundlach
Feb 15, 2022
Author

Thanks @MatthiasValvekens. Yesterday, while i'm searching for my problem i just saw it when passing by the documention.
I will check it out soon.

but unfortunately i've got a new Problem. Because of interprocess communication and our workflow design i have to import the trusted certificate from the enviroment. In Reference to the Documentation
https://github.com/MatthiasValvekens/certvalidator/blob/master/docs/api.md#validationcontext-class

ValidationContext() class
...
    :param trust_roots:
        If the operating system's trust list should not be used, instead
        pass a list of byte strings containing DER or PEM-encoded X.509
        certificates, or asn1crypto.x509.Certificate objects. These
        certificates will be used as the trust roots for the path being
        built.
...

i tried the following and failed:

gundlac@batch-ost:~$ export x_CERT_PEM_KEY="-----BEGIN CERTIFICATE-----
MIIC8TCCAnegAwIBAgIDD+SeMAoGCCqGSM49BAMDMF4xCzAJBgNVBAYTAkRFMRUw
EwYDVQQKEwxELVRydXN0IEdtYkgxHzAdBgNVBAMTFkQtVFJVU1QgUm9vdCBDQSAy
IDIwMTcxFzAVBgNVBGETDk5UUkRFLUhSQjc0MzQ2MB4XDTE3MDIwMTA4MzgyMVoX
DTMyMDIwMTA4MzgyMVowXjELMAkGA1UEBhMCREUxFTATBgNVBAoTDEQtVHJ1c3Qg
R21iSDEfMB0GA1UEAxMWRC1UUlVTVCBSb290IENBIDIgMjAxNzEXMBUGA1UEYRMO
TlRSREUtSFJCNzQzNDYwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAAS/S9pSYDmhDq98
J2rmX9uyf6uzuiY/uAiL5y7f2M3LQmqxNyavuWSOyNi++ByEru6tztgRiPvACHrU
5MoQoKHzxoh6jTojnAiDbrmCp6fVjm/UUFIJEjz5YD+tVkKKm9CjggEFMIIBATAP
BgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBQgPODzbyeRhmVhdHfWgQm3ZXyvHzAO
BgNVHQ8BAf8EBAMCAQYwgb4GA1UdHwSBtjCBszB0oHKgcIZubGRhcDovL2RpcmVj
dG9yeS5kLXRydXN0Lm5ldC9DTj1ELVRSVVNUJTIwUm9vdCUyMENBJTIwMiUyMDIw
MTcsTz1ELVRydXN0JTIwR21iSCxDPURFP2NlcnRpZmljYXRlcmV2b2NhdGlvbmxp
c3QwO6A5oDeGNWh0dHA6Ly9jcmwuZC10cnVzdC5uZXQvY3JsL2QtdHJ1c3Rfcm9v
dF9jYV8yXzIwMTcuY3JsMAoGCCqGSM49BAMDA2gAMGUCMQDvs2yHlmUW2xSB4BVE
soqIa2Ahq/e2S8enS8gQ8Jl9+OA6/TFENFuLkOa7MDkESakCMDOjUmEXrbmCDO0e
HZr/uD3sD3ZZzHWDu2oX2bV3Zdkgl1RN4TVRXJgKmZaT1TC4dw==
-----END CERTIFICATE-----"
gundlac@batch-ost:~$ python3
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pyhanko_certvalidator import ValidationContext
>>> import os
>>> vc = ValidationContext(trust_roots=[str.encode(os.getenv('x_CERT_PEM_KEY'))])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.8/dist-packages/pyhanko_certvalidator/context.py", line 502, in __init__
    self.certificate_registry = CertificateRegistry(
  File "/usr/local/lib/python3.8/dist-packages/pyhanko_certvalidator/registry.py", line 219, in __init__
    self.register(trust_root)
  File "/usr/local/lib/python3.8/dist-packages/pyhanko_certvalidator/registry.py", line 137, in register
    if cert.issuer_serial in self.certs:
AttributeError: 'bytes' object has no attribute 'issuer_serial'
>>> [str.encode(os.getenv('x_CERT_PEM_KEY'))]
[b'-----BEGIN CERTIFICATE-----\nMIIC8TCCAnegAwIBAgIDD+SeMAoGCCqGSM49BAMDMF4xCzAJBgNVBAYTAkRFMRUw\nEwYDVQQKEwxELVRydXN0IEdtYkgxHzAdBgNVBAMTFkQtVFJVU1QgUm9vdCBDQSAy\nIDIwMTcxFzAVBgNVBGETDk5UUkRFLUhSQjc0MzQ2MB4XDTE3MDIwMTA4MzgyMVoX\nDTMyMDIwMTA4MzgyMVowXjELMAkGA1UEBhMCREUxFTATBgNVBAoTDEQtVHJ1c3Qg\nR21iSDEfMB0GA1UEAxMWRC1UUlVTVCBSb290IENBIDIgMjAxNzEXMBUGA1UEYRMO\nTlRSREUtSFJCNzQzNDYwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAAS/S9pSYDmhDq98\nJ2rmX9uyf6uzuiY/uAiL5y7f2M3LQmqxNyavuWSOyNi++ByEru6tztgRiPvACHrU\n5MoQoKHzxoh6jTojnAiDbrmCp6fVjm/UUFIJEjz5YD+tVkKKm9CjggEFMIIBATAP\nBgNVHRMBAf8EBTADAQH/MB0GA1UdDgQWBBQgPODzbyeRhmVhdHfWgQm3ZXyvHzAO\nBgNVHQ8BAf8EBAMCAQYwgb4GA1UdHwSBtjCBszB0oHKgcIZubGRhcDovL2RpcmVj\ndG9yeS5kLXRydXN0Lm5ldC9DTj1ELVRSVVNUJTIwUm9vdCUyMENBJTIwMiUyMDIw\nMTcsTz1ELVRydXN0JTIwR21iSCxDPURFP2NlcnRpZmljYXRlcmV2b2NhdGlvbmxp\nc3QwO6A5oDeGNWh0dHA6Ly9jcmwuZC10cnVzdC5uZXQvY3JsL2QtdHJ1c3Rfcm9v\ndF9jYV8yXzIwMTcuY3JsMAoGCCqGSM49BAMDA2gAMGUCMQDvs2yHlmUW2xSB4BVE\nsoqIa2Ahq/e2S8enS8gQ8Jl9+OA6/TFENFuLkOa7MDkESakCMDOjUmEXrbmCDO0e\nHZr/uD3sD3ZZzHWDu2oX2bV3Zdkgl1RN4TVRXJgKmZaT1TC4dw==\n-----END CERTIFICATE-----']

Same way works in cryptography.hazmat.primitives.serialization.load_pem_public_key and is there documented similarly.
Parameters data (bytes) – The PEM encoded key data.

Seems ValidationContext expect an object with the attribut issuer_serial. From the code that should be .authority.TrustAnchor or, as documented, asn1crypto.x509.Certificate. I cannot figure out how i can create an .authority.TrustAnchor Object from a byte encoded PEM String. Do you have a hint?

Best Regards,

Thomas

2 replies

thomasgundlach Feb 15, 2022
Author

@MatthiasValvekens
Okaj, i have a solution and untested for further code:

import os
from asn1crypto import x509, pem
from pyhanko_certvalidator import ValidationContext
 _, _, digicert_ca_bytes = pem.unarmor(str.encode(os.getenv('x_CERT_PEM_KEY')))
vc = ValidationContext(trust_roots=[x509.Certificate.load(digicert_ca_bytes)])

But just for interesting: Do you know a way how i can put a list of byte strings containing PEM-encoded X.509 into ValidationContext?

MatthiasValvekens Feb 15, 2022
Maintainer

Two things:

pyhanko-certvalidator is currently going through major refactoring, so best not to pay attention to what happens on the master branch for now... Better to switch the view to the release tag you're using.
Since the addition of attribute certificate support in 0.19.0, I removed the ability to pass in certificates as raw bytes, since otherwise heuristics would be required to determine whether something is a "regular" certificate or an attribute cert (in some places). This is mentioned in the release notes but it seems that I forgot to update the pyhanko-certvalidator docs... Sorry about that.

So decoding the bytes using pem.unarmor and loading them using x509.Certificate.load(...) is indeed the way to go.

thomasgundlach · 2022-02-18T15:06:21Z

thomasgundlach
Feb 18, 2022
Author

Hej @MatthiasValvekens ,

works perfect. Thank you for making that clear!

I've got another Problem. But i will open a new thread for that.

Best Regards,

Thomas

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CLI: Trouble to verfiy signed pdfs with pfx certificate store #77

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 4 comments 3 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

CLI: Trouble to verfiy signed pdfs with pfx certificate store #77

thomasgundlach Feb 14, 2022

Replies: 4 comments · 3 replies

MatthiasValvekens Feb 14, 2022 Maintainer

thomasgundlach Feb 15, 2022 Author

MatthiasValvekens Feb 15, 2022 Maintainer

thomasgundlach Feb 15, 2022 Author

thomasgundlach Feb 15, 2022 Author

MatthiasValvekens Feb 15, 2022 Maintainer

thomasgundlach Feb 18, 2022 Author

thomasgundlach
Feb 14, 2022

Replies: 4 comments 3 replies

MatthiasValvekens
Feb 14, 2022
Maintainer

thomasgundlach
Feb 15, 2022
Author

MatthiasValvekens Feb 15, 2022
Maintainer

thomasgundlach
Feb 15, 2022
Author

thomasgundlach Feb 15, 2022
Author

MatthiasValvekens Feb 15, 2022
Maintainer

thomasgundlach
Feb 18, 2022
Author