Skip to content

[4.1.0] Regresion or Undocumented Behavioral change in the X509_check_host #33420

Open

Description

@tregubovav-dev

Problem description

Starting the 4.1.0 the X509_check_host always fails when the X509 certificate contains the only subject DN with CN identifies the hostname and no SAN(s) is included. and no flags are used (flags parameter is set to 0).
The call like that X509_check_host(cert, expected_hostname, 0, 0, NULL); returns 0 in Openssl 4.1.0 and returns 1 with previous releases.

The openssl verify command clearly display the behaviour difference. The command below returns:

openssl verify -CAfile <CAFilePath> -verify_hostname <`CN entry` in the `Subject DN`> <certificate path> 
  • In 4.1.0:
<Subject DN content>
error 62 at 0 depth lookup: hostname mismatch
error <certificate path>: verification failed
  • in 4.0.2 and earlier:
    <certificate path>: OK

Steps to reproduce

  • Generate the certificate with the FQDN/hostname in the Subject DN and does not contain any Subject Alternative Name fields.
  • run the command below against the 4.1.0 and earlier version(s) of openssl binaries,

Activity

  1. esyr commented on Oct 27, 2026

    @esyr
    Member
  2. tregubovav-dev commented on Oct 27, 2026

    @tregubovav-dev
    Author
    Certificate Description
    ca.crt.txt CA Certificate
    server_ecdsa_p256.crt.txt EC-curve certificate
    server_ed25519.crt.txt ED-curve certificate
    server_rsa.crt.txt: RCA certificate

    I've added the example files for the reproduction.
    Certificates Subject DN is Subject: CN = srv.test.tld
    Use:
    openssl verify -CAfile ca.crt.txt -verify_hostname srv.test.tld <certificate_filename>

  3. idrassi commented on Oct 28, 2026

    @idrassi
    Member

    Thanks for the certificates. I have reproduced the issue.

    PR #31982 intentionally changed this behavior. The changelog documents it, but the command line manual still promises CN matching. The standard verification options don't expose a switch to enable it. For these certificates, reissuing with DNS:srv.test.tld in the SAN is the straightforward migration.

    Applications that perform their own CN matching also need to be updated. OpenSSL 4.1 applies DNS constraints to the CN only with X509_CHECK_FLAG_ALWAYS_CHECK_SUBJECT set and X509_CHECK_FLAG_NEVER_CHECK_SUBJECT clear. An application can otherwise accept a CN that OpenSSL hasn't checked against the issuer's DNS constraints.

    Curl is one example: it relies on OpenSSL for chain validation but performs CN fallback itself. I tested curl commit 3163ec065643, from tag curl-8_23_0, against both releases.

    Only the root was trusted. The server supplied this chain beneath it:

    Intermediate: critical nameConstraints, permitted DNS:allowed.test
    Leaf: CN=victim.test, no subjectAltName
    

    I requested https://victim.test over loopback using --resolve, with certificate and hostname verification enabled.

    OpenSSL curl result
    4.0.2 Error 60; OpenSSL error 47, permitted subtree violation
    4.1.0 Exit 0; HTTP response received

    Reversing the x509_vfy.c change in e7f3a42 restores rejection without restoring CN matching by default. That isolates the cause: it doesn't settle where the compatibility fix belongs. OpenSSL integrated hostname verification rejects this certificate.

    For applications retaining independent CN matching, the extra checks must be enabled during chain verification. Setting ALWAYS_CHECK_SUBJECT there rejects this example in my probe. Adding it only to a later X509_check_host call won't restore those checks.

    The flag isn't an exact replacement for the old behavior:

    • In OpenSSL hostname matcher, it lets a matching CN override a DNS SAN mismatch.
    • With an independent matcher, it doesn't change the application SAN precedence but can reject an otherwise acceptable certificate because an unused CN fails the additional constraint check.

    @t8m The curl example shows that moving to 4.1 can weaken certificate validation in applications that retain their own CN matching. Could the project publish a prominent migration notice explaining this impact and how applications should enable the necessary CN constraint checks?

  4. tregubovav-dev commented on Oct 28, 2026

    @tregubovav-dev
    Author

    Thanks a lot Mounir to pointing me out the PR and changelog that missed from my eyes.
    It finally confirms our approach to resolve this behavior change.

    I would suggest to update the openssl tool documentation update to avoid customers' confusion.
    You are free to close this issue if you are not planning do any action based on it.

  5. idrassi commented on Oct 28, 2026

    @idrassi
    Member

    There is still the impact I described above on certificate validation for applications like curl.
    We need to communicate about it to avoid security issues on third party application that would update to 4.1. Not sure how this can be done.

  6. added
    help wantedIssue to be resolved or being resolved by community.
    triaged: bugThe issue/pr is/fixes a bug
    and removed
    issue: bug reportThe issue was opened to report a bug
    on Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    help wantedIssue to be resolved or being resolved by community.triaged: bugThe issue/pr is/fixes a bug

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions