Use of Belgian ID Card (Windows OS)

[AlainCNpt]

I'm trying to make a Proof of Concept on for signing with the bEID-Card.

During the signing process, I get errors when I try to connect to a timestamper, but I think this is secondary, as without it, I'm not able to sign the document either ... On one hand it looks required (in the Metadata), but when signing with Acrobat, a TSA server isn't used either)

When testing the signed document with the pyHanko library itself, I get the error;
"ValueError: Error parsing asn1crypto.cms.ContentInfo - method should have been constructed, but primitive was found"
Looking for "primitive" and "constructed" in the documentation, I don't really find anything that gives me a hint of what I'm doing wrong...
Anyone can point me in the right direction?

When adding TSA during signing, it looks like additional certificates are needed.
pyhanko_certvalidator.errors.PathBuildingError: Unable to build a validation path for the certificate "Common Name: Timestamp Unit, Serial Number: 202101, Organizational Unit: QTSP: FPS Policy and Support - BOSA (NTRBE-0671516647), Organization: Kingdom of Belgium - Federal Government, Locality: Brussels, Country: BE" - no issuer matching "Common Name: Belgium Root CA6; Organizational Unit: FPS Policy and Support - BOSA (NTRBE-0671516647), FPS Home Affairs - BIK-GCI (NTRBE-0362475538); Organization: Kingdom of Belgium - Federal Government; Locality: Brussels; Country: BE" was found

The basics of my tests (although there isn't much more, I added the full code in attachment):
`
#Signer => also used for validation only (to build ValidationContext)
useDLL = r"C:\Windows\System32\beidpkcs11.dll"
#useDLL = r"C:\Windows\System32\beid_ff_pkcs11.dll"

sess = open_beid_session(useDLL)
beid = BEIDSigner(sess,False)

#Validation Context
vcBEID = ValidationContext(trust_roots=[beid.signing_cert], allow_fetching=True)

#Timestamp client -- Doesn't seem to work yet ... which certicates to add?
tsa_client = timestamps.HTTPTimeStamper(' http://tsa.belgium.be/connect')

filenameUnsigned = r'c:\python\PyHanko\Document_Unsigned.pdf'
filenameSigned = r'c:\python\PyHanko\Document_Signed.pdf'

with open(filenameUnsigned, 'rb') as docUnsigned:
w = IncrementalPdfFileWriter(docUnsigned)
r = PdfFileReader(docUnsigned)

try:
    signature_meta = signers.PdfSignatureMetadata(
            field_name='Dig. Handtekening', 
            md_algorithm='sha256',
            # location='Belgium',
            # reason='tbd',
            name=beid.subject_name, # Probably not required when signature is correct
            certify=False,          # Default :False --- True voor Author (Max 1/Doc)
            subfilter=SigSeedSubFilter.PADES,                            
            embed_validation_info=True,  # Embed OCSP / CRLs
            # use_pades_lta=False, # Probably to be set tot True when TSA works .. or is it always necessary?
            # timestamp_field_name: typing.Optional[str] = None,
            
            validation_context=vcBEID,
            #docmdp_permissions=MDPPerm.NO_CHANGES, # Default: MDPPerm.FILL_FORMS, => only for certify = true                
    )

    out = signers.sign_pdf(w, signature_meta, signer=beid) # , timestamper=tsa_client)

    docSigned = open(filenameSigned, 'wb')
    w.write(docSigned)
    docSigned.close()

except PKCS11Error as err:
    print("PKCS11Error: {0}".format(err))

docUnsigned.close()

`
testBEID2.txt

[MatthiasValvekens]

Hi @AlainCNpt

That particular error likely arises because you're trying to decode malformed ASN.1 data somewhere (constructed/primitive are ASN.1 terminology). I'd need to see some sample data or at least a stack trace from asn1crypto to figure out why that happens.

Regardless, there are a couple things that stand out in your code:

• You're instantiating the validation context with your signing certificate as a trust root. Trust roots (by definition) are trusted by fiat, and not subject to revocation. Therefore the CRL and OCSP fetches won't do anything useful. You'll probably want to install the government trust roots on your system and use the system trust, or (depending on how and where you'll deploy this) you might be OK with extracting the trust root + intermediate certs from the card. Either way, the relevant root certs should be available from https://repository.eid.belgium.be.

• The reason why timestamping fails is similar: there's no trust root around. Note that depending on the age of your card, the applicable root certificate for your ID card may not be the same as the one used by the government's public TSA.

• You can't have periods in (non-nested) field names in PDF. I was under the impression that I made pyHanko throw an error in those cases, but apparently I didn't. Anyway, if it doesn't fail with an error prior to signing, that'd definitely be a bug that I would have to look into. In the meantime, you'll probably want to change that field name :)

All that being said, I don't know for sure what the root cause of the ASN.1 parse error would be.
I haven't had the time yet to try and actually run your code; I'll try to do that some time in the next couple of days. Hopefully that'll clear things up.

Final question: how recent is your eID? Do you have one of the newer 1.8 cards (the ones that use ECDSA)? It shouldn't matter, but not having access to one of those, I never actually tried using them with pyHanko, so you never know...

Anyway, hopefully that already helps somewhat.

[AlainCNpt]

HI Matthias,

Thanks for the reply. I tried to get any further, but I wasn't very successful. I exported the Belgium Root CA, and the Citizen CA from the card, but adding them - one at the time - in the ValidationContext didn't make improvements.
Anyway, for reading purposes, I removed the trusted_roots (in another test), and this worked ...

I also tried to make some changes with the Signature-metadata, but I didn't notice anything that looked/worked better.

The fieldname was changed as well, but the period didn't bother Acrobat Reader (but it still isn't best practice to work this way).

My card was issued in 2017, so not that recent.

The rest of the week I can't make priorities on this anymore, if you feel like verifying if you get better results, I add my testdata.
PyHanko.zip

I made a version with my normal way of adding signature fields, and one based on your library, to make sure the field isn't broken at the first place...

Thanks,
Alain

[AlainCNpt]

Didn't find that much time yet, but tried to extract the root certificate from the card ... It doesn't raise errors this way (when signing), but still no luck getting a valid signature (not even an invalid one when using a pyHanko-reader)

The part to read all/one certificates:
`
##trusts = []
##for installedCerts in sess.get_objects({Attribute.CLASS: ObjectClass.CERTIFICATE}):
###trusts.append(x509.Certificate.load(installedCerts[Attribute.VALUE]))

certRoot = sess.get_key(object_class=ObjectClass.CERTIFICATE, label='Root')
trusts = [x509.Certificate.load(certRoot[Attribute.VALUE])]

##Validation Context
vcBEID = ValidationContext(trust_roots=trusts,allow_fetching=True)
`

[MatthiasValvekens]

Hi @AlainCNpt, I finally found the time to look at your actual files (thanks for sending those along!) and the problem is now obvious---sorry I didn't spot it earlier...

The signature container in your signed file is zeroed out. The reason why that is the case is because you're overwriting the content of the signed file with the contents of the incremental updater in its state prior to the signature being produced.

In other words, these lines are the problem:

 out = signers.sign_pdf(w, signature_meta, signer=beid) # , timestamper=tsa_client)

docSigned = open(filenameSigned, 'wb')
w.write(docSigned)
docSigned.close()

In actuality, the output you want is in out, not in the file pointed to by docSigned. You'll want to replace that snippet with something like this:

with open(filenameSigned, 'wb') as outf:
    signers.sign_pdf(w, signature_meta, signer=beid, output=outf)

or alternatively

out =  signers.sign_pdf(w, signature_meta, signer=beid)
with open(filenameSigned, 'wb') as outf:
    buf = out.getbuffer()
    outf.write(buf)
    buf.release()

Sorry, I should really have noticed that mistake the first time around. For future reference: the rule of thumb is that the content of a PDF writer is no longer relevant after you pass it through sign_pdf.

[AlainCNpt]

Hi Matthias,

Thanks for all the time you spend on noob-questions like these ... I used the reader first - in my trials - to detect the unsigned signature fields (only possible with the reader), and then tried to attempt to sign them (only possible with the writer), so for this I mixed those, which will be needed in the end, and in the end messed up with the output to file ...

The signing is working now.. not sure if I will find time to implement the time-service in a short time, but I can continue now :)

Thanks,
Alain

[AlainCNpt]

Update on the timestamp:
I added this, with the sectigo-tsa server (not sure whether I'm allowed to use this in a commercial setting, but as POC this works, I contacted them already). I tried freetsa as well, but this had issues with the trust on Acrobat.

Changes to the script:
(has to be placed before the creation of the validationContext)
## https://www.tbs-certificates.co.uk/FAQ/en/SectigoQualifiedTimeStampingRootR45.html
EXTERNAL_TSA_URL = 'http://timestamp.sectigo.com/qualified'
for cert in load_certs_from_pemder([r'c:\python\PyHanko\SectigoQualifiedTimeStampingRootR45.crt']):
trusts.append(cert)
tsa_client = timestamps.HTTPTimeStamper(EXTERNAL_TSA_URL, timeout=30)

And finally:
out = signers.sign_pdf(w, signature_meta, signer=beid, timestamper=tsa_client)

Verifying with acrobat it shows that the "Signature includes an embedded timestamp" (i.s.o. "clock on the signer's computer")

But it's still not LTV enabled... In what direction do I need to look for that?

[AlainCNpt]

It is now "LTV enabled" according to Acrobat Reader, but the LTV-check on PyHanko isn't as happy ... I added an extra Validation Context, but probably need to implement the Revocation Info ... but this is not important at the moment, I can live with this

I add my current script for people searching a start... I "stole" a lot from other discussions as well :)
testBEID.py.txt

Next step on my plan (but probably only in a few weeks): Could multiple documents be signed with just entering the pincode once...

[MatthiasValvekens]

If you're sure you embedded all the necessary revocation data (which pyHanko should do if properly configured), then chances are your problem is that your pyHanko validation context doesn't include all the trust roots you need, or is missing some intermediate certs. Either way, pinning down what Acrobat calls "LTV enabled" is not totally clear. In some respects, they're not strict enough (e.g. sometimes non-timestamped signatures with revocation info are still considered OK), but the converse can certainly happen as well. There are also different ways of embedding revocation info that might cause subtly different behaviour in viewers...

Bottom line: there are standards on how to perform validation over long timescales (part of AdES), but there's no real way to definitively classify a signature as "LTV enabled" or not. As such, Acrobat's LTV marker is necessarily a bit of a moving target.

EDIT: I'll also add that pyHanko's LTV-style validators are somewhat overly dogmatic and don't really follow any particular standard. Doing something about that is on the roadmap, but it's not that straightforward, unfortunately.

Next step on my plan (but probably only in a few weeks): Could multiple documents be signed with just entering the pincode once...

I'm almost completely sure that the software on the ID card chip requires the user to reauthenticate for every signing operation, even for multiple operations within the same PKCS#11 session. Having it any other way would be problematic, since it would allow unscrupulous actors to leverage "cached" sessions to sign documents behind the user's back.

The restrictions around the authentication key might be a bit more lax, but that won't help you if you're trying to sign PDF documents (well, if you want the signatures to have any sort of legal value, that is ;) ).