Review follow-up for F-5621. wolfSSL_check_domain_name() only matches
DNS names: on this path CheckForAltNames() is called with isIP=0, so
iPAddress SANs are always skipped (verified on v5.8.4-stable and
master). The default invocation (host 127.0.0.1) therefore failed the
handshake with DOMAIN_NAME_MISMATCH (-322) once hostname verification
was enabled by default.
Skip the hostname check for IP literal hosts and say so, keeping
CERT_REQUIRED verification. This also stops offering an IP literal in
SNI, which RFC 6066 forbids. Connecting by DNS name still enables the
hostname check.
Add unit tests for the IP literal paths and an end-to-end test that
runs server.py and client.py with default arguments.
The client example set CERT_REQUIRED and loaded CA roots but never set
check_hostname or passed server_hostname to wrap_socket, so wolfSSL
validated the chain to a trusted CA without binding the certificate to
the requested host. A peer presenting any CA-trusted certificate for a
different hostname would be accepted by anyone reusing this as a secure
client template. Make verification configure hostname checking by
default (via a new configure_verification helper) and add a -n flag to
opt out explicitly for IP literals or test certificates.
Kareem
Juliusz Sosinowicz