Self signed cert not working, getting error:

[samholst]

Generated a new self-signed cert using:

openssl req -x509 -newkey rsa:4096 -keyout private_key.pem -out cert.pem -days 365 -nodes

Which creates:

cert.pem
private_key.pem

Then I have a test pdf located in the current directory I am in in the console and I use this command:

pyhanko sign addsig --field Sig1 pemder --no-pass --cert cert.pem --key private_key.pem SignedLegal.pdf signed_output.pdf

This then produces an error:

2025-01-02 12:12:23,955 - cli - ERROR - Generic processing error.
Error: Generic processing error.

Am I missing something? @MatthiasValvekens

[MatthiasValvekens]

Impossible to say without more info; you should rerun with pyhanko --verbose. Also, try signing some other files to figure out if the issue is with the input document or with the key. If it's a problem with the input document, diagnosing it will be (all but) impossible without the actual file.

[samholst]

test_pyhanko.zip

All passwords are test in the .zip. and the pdf file is test_sig.pdf. The pdf has one sentence of wording, very small test file.

pfx generated from following code:

openssl genrsa -des3 -out ./local_dev.key 2048
openssl req -key ./local_dev.key -new -x509 -days 365 -out ./local_dev.crt
openssl pkcs12 -inkey ./local_dev.key -in ./local_dev.crt -export -out ./local_dev.pfx
openssl base64 -in ./local_dev.pfx -out ./base64_pfx

pyhanko --verbose sign addsig --field Sig1 pemder --cert cert.pem --key private_key.pem test_sig.pdf signed_output.pdf
2025-01-04 21:40:54,008 - root - DEBUG - Running with --verbose
2025-01-04 21:40:54,008 - root - DEBUG - There was no configuration to parse.
Key passphrase:
2025-01-04 21:40:56,060 - asyncio - DEBUG - Using selector: KqueueSelector
2025-01-04 21:40:56,061 - tzlocal - DEBUG - /etc/localtime found
2025-01-04 21:40:56,066 - tzlocal - DEBUG - 1 found:
 {'/etc/localtime is a symlink to': 'America/Los_Angeles'}
2025-01-04 21:40:56,332 - cli - ERROR - Generic processing error.
Traceback (most recent call last):
  File "/Users/sam/.pyenv/versions/pypy3.10-7.3.17/lib/pypy3.10/site-packages/pyhanko/cli/commands/signing/plugin.py", line 132, in _callback
    _callback_logic(plugin, infile, outfile, **kwargs)
  File "/Users/sam/.pyenv/versions/pypy3.10-7.3.17/lib/pypy3.10/site-packages/pyhanko/cli/commands/signing/plugin.py", line 111, in _callback_logic
    result = PdfSigner(
  File "/Users/sam/.pyenv/versions/pypy3.10-7.3.17/lib/pypy3.10/site-packages/pyhanko/sign/signers/pdf_signer.py", line 1528, in sign_pdf
    result = asyncio.run(
  File "/Users/sam/.pyenv/versions/pypy3.10-7.3.17/lib/pypy3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/Users/sam/.pyenv/versions/pypy3.10-7.3.17/lib/pypy3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "/Users/sam/.pyenv/versions/pypy3.10-7.3.17/lib/pypy3.10/asyncio/futures.py", line 201, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "/Users/sam/.pyenv/versions/pypy3.10-7.3.17/lib/pypy3.10/asyncio/tasks.py", line 232, in __step
    result = coro.send(None)
  File "/Users/sam/.pyenv/versions/pypy3.10-7.3.17/lib/pypy3.10/site-packages/pyhanko/sign/signers/pdf_signer.py", line 1603, in async_sign_pdf
    prepared_br_digest, res_output = tbs_document.digest_tbs_document(
  File "/Users/sam/.pyenv/versions/pypy3.10-7.3.17/lib/pypy3.10/site-packages/pyhanko/sign/signers/pdf_signer.py", line 2607, in digest_tbs_document
    return self.cms_writer.send(
  File "/Users/sam/.pyenv/versions/pypy3.10-7.3.17/lib/pypy3.10/site-packages/pyhanko/sign/signers/cms_embedder.py", line 493, in write_cms
    yield from sig_obj.fill(
  File "/Users/sam/.pyenv/versions/pypy3.10-7.3.17/lib/pypy3.10/site-packages/pyhanko/sign/signers/pdf_byterange.py", line 304, in fill
    md.update(output_buffer[:sig_start])
TypeError: argument 'data': from_buffer() got a 'memoryview' object, which supports the buffer interface but cannot be rendered as a plain raw address on PyPy
Error: Generic processing error.

[MatthiasValvekens]

TypeError: argument 'data': from_buffer() got a 'memoryview' object, which supports the buffer interface but cannot be rendered as a plain raw address on PyPy

Ah, you're running on PyPy, which is not a platform that pyHanko currently supports. If you know how to address this discrepancy you're welcome to file a PR (I'm willing to do some degree of legwork to figure out how to regression test it), but I don't have the (time) resources to deal with this myself.

Is switching to CPython an option for you?

[samholst]

Yes, thank you for pointing that out! I install it. It did end up signing it without any errors with a simple sign, but more complex on which I'd want with options to validate after the expires fails, but when I try to verify it outputs an error:

pyhanko.yml

signers:
  testsetup:
    key: "/Users/sam/pyhanko/test/private_key.pem"
    cert: "/Users/sam/pyhanko/test/cert.pem"

pemder-setups:
  testsetup:
    key-file: /Users/sam/pyhanko/test/private_key.pem
    cert-file: /Users/sam/pyhanko/test/cert.pem
    # other-certs: path/to/more/certs.chain.pem
    key-passphrase: test

validation-contexts:
    testsetup:
        trust: /Users/sam/pyhanko/test/cert.pem
        trust-replace: true

Signing without --timestamp-url https://freetsa.org/tsr --with-validation-info --validation-context testsetup --use-pades works:

pyhanko --verbose sign addsig --field Sig1 pemder --cert cert.pem --key private_key.pem test_sig.pdf signed_output.pdf
2025-01-10 08:53:28,469 - root - DEBUG - Running with --verbose
2025-01-10 08:53:28,469 - root - DEBUG - Finished reading configuration from pyhanko.yml.
Key passphrase:
2025-01-10 08:53:30,359 - asyncio - DEBUG - Using selector: KqueueSelector
2025-01-10 08:53:30,359 - tzlocal - DEBUG - /etc/localtime found
2025-01-10 08:53:30,360 - tzlocal - DEBUG - 1 found:
 {'/etc/localtime is a symlink to': 'America/Los_Angeles'}

I'm guessing due to self sign this will fail:

pyhanko --verbose sign addsig --field Sig1 --timestamp-url https://freetsa.org/tsr \
    --with-validation-info --validation-context testsetup \
    --use-pades pemder --cert cert.pem --key private_key.pem test_sig.pdf signed_output.pdf

2025-01-10 08:49:26,796 - root - DEBUG - Running with --verbose
2025-01-10 08:49:26,797 - root - DEBUG - Finished reading configuration from pyhanko.yml.
Key passphrase:
2025-01-10 08:49:28,926 - asyncio - DEBUG - Using selector: KqueueSelector
2025-01-10 08:49:28,927 - tzlocal - DEBUG - /etc/localtime found
2025-01-10 08:49:28,927 - tzlocal - DEBUG - 1 found:
 {'/etc/localtime is a symlink to': 'America/Los_Angeles'}
2025-01-10 08:49:28,928 - pyhanko_certvalidator.fetchers.requests_fetchers.cert_fetch_client - INFO - Retrieving issuer certs for Common Name: test, Organization: test, Locality: test, State/Province: test, Country: te...
2025-01-10 08:49:28,928 - cli - ERROR - Generic processing error.
Traceback (most recent call last):
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/site-packages/pyhanko_certvalidator/__init__.py", line 34, in find_valid_path
    async for candidate_path in paths:
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/site-packages/pyhanko_certvalidator/registry.py", line 649, in __anext__
    raise PathBuildingError(
pyhanko_certvalidator.errors.PathBuildingError: Unable to build a validation path for the certificate "Common Name: test, Organization: test, Locality: test, State/Province: test, Country: te" - no issuer matching "Common Name: test, Organization: test, Locality: test, State/Province: test, Country: te" was found

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/site-packages/pyhanko/cli/runtime.py", line 50, in pyhanko_exception_manager
    yield
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/site-packages/pyhanko/cli/commands/signing/plugin.py", line 132, in _callback
    _callback_logic(plugin, infile, outfile, **kwargs)
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/site-packages/pyhanko/cli/commands/signing/plugin.py", line 117, in _callback_logic
    ).sign_pdf(
      ^^^^^^^^^
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/site-packages/pyhanko/sign/signers/pdf_signer.py", line 1528, in sign_pdf
    result = asyncio.run(
             ^^^^^^^^^^^^
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/site-packages/pyhanko/sign/signers/pdf_signer.py", line 1590, in async_sign_pdf
    validation_info = await signing_session.perform_presign_validation(
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/site-packages/pyhanko/sign/signers/pdf_signer.py", line 1773, in perform_presign_validation
    signer_path = await self._perform_presign_signer_validation(
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/site-packages/pyhanko/sign/signers/pdf_signer.py", line 1872, in _perform_presign_signer_validation
    signer_cert_validation_path = await validator.async_validate_usage(
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/site-packages/pyhanko_certvalidator/__init__.py", line 269, in async_validate_usage
    validated_path = await self.async_validate_path()
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/site-packages/pyhanko_certvalidator/__init__.py", line 136, in async_validate_path
    self._path = candidate_path = await find_valid_path(
                                  ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/sam/.pyenv/versions/3.12.6/lib/python3.12/site-packages/pyhanko_certvalidator/__init__.py", line 44, in find_valid_path
    raise InvalidCertificateError(
pyhanko_certvalidator.errors.InvalidCertificateError: The X.509 certificate provided is self-signed - "Common Name: test, Organization: test, Locality: test, State/Province: test, Country: te"
Error: Generic processing error.

pyhanko sign validate --pretty-print signed_output.pdf

2025-01-10 08:45:44,537 - pyhanko.sign.validation.generic_cms - WARNING - The active key usage policy requires at least one of the key usage extensions non repudiation to be present.
=============
Field 1: Sig1
=============


Signer info
-----------
Certificate subject: "Common Name: test, Organization: test, Locality: test, State/Province: test, Country: te"
Certificate SHA1 fingerprint: 8c6dce47c131f73a77b955c45db085213615dbc0
Certificate SHA256 fingerprint: 9fdfa659541f72941f59ff71a79e4e81a44a4c5ebd0a176827e42f97c8183638
Trust anchor: "No path to trust anchor found."
The signer's certificate is untrusted.


Integrity
---------
The signature is cryptographically sound.

The digest algorithm used was 'sha512'.
The signature mechanism used was 'sha512_rsa'.


Signing time
------------
Signing time as reported by signer: 2025-01-06T07:26:35+00:00


Modifications
-------------
The signature covers the entire file.


Bottom line
-----------
The signature is judged INVALID.


Error: Validation failed

Any pointers?

I have tried using --trust and one other flag I can't find at the moment, but I'm most likely missing the correct setup due to my lack of knowledge. I've been reading the guide PDF on pyhanko's site with all the options and can't find anything yet to resolve the error.