Add unittest for different openssl versions

[drwetter]

This adds a unit test to compare a run against google with the supplied openssl version vs /usr/bin/openssl .

This would fix #2626.

It looks like there are still points to clarify

• NPN output is different (bug)
• Newer openssl version claims it's ECDH 253 instead of ECDH 256.
• Newer openssl version claims for 130x cipher it's ECDH 253, via sockets it´s ECDH/MLKEM. This seems a bug (@dcooper16)

A todo is also restricing the unit test to the one where openssl is being used. E.g. the ROBOT check and more aren't done with openssl. So there's no value checking this here.

What is your pull request about?

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Typo fix
• Documentation update
• Update of other files

If it's a code change please check the boxes which are applicable

• For the main program: My edits contain no tabs and the indentation is five spaces
• I've read CONTRIBUTING.md and Coding_Convention.md
• I have tested this fix against >=2 hosts and I couldn't spot a problem
• I have tested this new feature against >=2 hosts which show this feature and >=2 host which does not (in order to avoid side effects) . I couldn't spot a problem
• For the new feature I have made corresponding changes to the documentation and / or to help()
• If it's a bigger change: I added myself to CREDITS.md (alphabetical order) and the change to CHANGELOG.md

[drwetter]

[dcooper16]

• NPN output is different (bug)

I noticed the documentation for s_client says that "The flag -nextprotoneg cannot be specified if -tls1_3 is used.." So, I tried running the $OPENSSL s_client test from run_npn() with a -no_tls1_3 flag added and that seems to provide the desired results. Perhaps this flag needs to be added whenever testing with a version of OpenSSL that supports TLS 1.3.

• Newer openssl version claims it's ECDH 253 instead of ECDH 256.
• Newer openssl version claims for 130x cipher it's ECDH 253, via sockets it´s ECDH/MLKEM. This seems a bug (@dcooper16)

I believe this is an issue that has come up before. The information in the KeyExch. column has only limited meaning since it reports what key exchange the server happened to select for one particular connection. In this case, the TLS 1.3 ciphers are being detected by OpenSSL in one case and using tls_sockets() in the other. When tls_sockets() is used, the server chooses its preferred option of X25519MLKEM768. When OpenSSL is used, that option is not offered by the client, and so the server chooses X25519. For TLs 1.2 and earlier, the ciphers are detected using OpenSSL in both cases, but only the newer OpenSSL supports X25519. So, the server using X25519 with the newer OpenSSL and P-256 with the older one.

The only way to ensure consistent results is to always use tls_sockets(), which would make testssl.sh slower. However, it is not clear that would be of substantial benefit, given that the KeyExch. a user actually sees depends on the list of options they provide the server and may depend on the order the items are listed in the ClientHello. This may be related to #2488, but it seems unnecessary as the list of all supported key exchange options is provided in the forward secrecy section.

[dcooper16]

FYI: When I test against google.com with /usr/bin/openssl on my computer I see some additional differences since my work computer is configured to disable options that are not NIST approved.

• While /usr/bin/openssl seems to support TLS 1 and TLS 1.1, it cannot connect with these protocols. This causes the TLS_FALLBACK_SCSV check to fail. . Ticketbleed testing also fails. Ticketbleed correctly reports "not vulnerable," but adds on "no session tickets." This seems to be because sub_session_ticket_tls() tries to connect to the server using TLS 1 and testssl.sh doesn't notice that the connect attempt failed.
• The ROBOT check fails at around line 20039, since the locked down version of OpenSSL rejects the -pkeyopt rsa_padding_mode:none option
• The locked-down version also reports different key exchange information in server preferences since X25519 and TLS_CHACHA20_POLY1305_SHA256 are disabled.

[drwetter]

the NIST approved issue should only be relevant for testing (yours, unfortunately) but not for CI.

[drwetter]

For your comment above : Are you referring to ECDH 253 vs ECDH/MLKEM only or also to ECDH 253 vs ECDH 256?

I remember we had a discussion about the latter, maybe also in the context of unit tests but I wasn't able to find it.

Personnally I need to sleep over it whether for consistency reasons we want to provide to the users the same result, independent on the openssl version used. Or whether we should live with that and for the sake of this test we just do some editing with perl.

[dcooper16]

For your comment above : Are you referring to ECDH 253 vs ECDH/MLKEM only or also to ECDH 253 vs ECDH 256?

Both. The table below shows the group that the server will use depending on the TLS protocol version and the client that sends the ClientHello. testssl.sh uses the results from $OPENSSL whenever $OPENSSL supports the protocol and cipher suite. So, for TLS 1 - TLS 1.2, the results are based on the difference between OpenSSL 1.0.2-chacha and /usr/bin/openssl. For TLS 1.3, the results are based on the difference between /usr/bin/openssl and tls_sockets().

I remember we had a discussion about the latter, maybe also in the context of unit tests but I wasn't able to find it.

It's really the same thing in both cases -- just the server choosing different groups depending on what is offered in the ClientHello. It just looks more different now since the output for the KeyExch. column looks different for X25519MLKEM768 than for P-256 or X25519.

Protocol	OpenSSL 1.0.2	OpenSSL 3.0.2	tls_sockets()
TLS 1 - TLS 1.2	P-256	X25519	X25519
TLS 1.3		X25519	X25519MLKEM768

[drwetter]

It's really the same thing in both cases

thanks for clarifying