Web Security, Part 4: TLS in Action

The TLS 1.3 handshake record by record — the key agreement, the certificate and every check a browser or curl makes on it — then the Basic-auth password inside the encrypted connection, with an eavesdropper under the wire showing what a copy of each record reveals. Self-signed, expired, wrongly named and stolen certificates, curl's -k and --cacert, and the browser's Proceed.

TLS is what turns the password of Basic HTTP Authentication from readable text into ciphertext.

Before any HTTP is sent, the client and the server agree on a secret that nobody on the path can compute.
The server proves that it holds the private key of a certificate issued for the name the client typed.
Every record of that handshake is shown as the two ends see it, and as an eavesdropper copying the wire sees it.
TLS Concepts comes before it: the same handshake, with the same two CAs, as keys and stamps instead of bytes.

Reading the drawing

The client asks for https://shop.example/orders with alice's password in a Basic Authorization header. curl adds -v, -k and --cacert self.crt, as in the course lab.
The server picks the certificate it presents, from a CA-issued one to the real certificate in the hands of an impostor without its private key.
The wire shows the last record whole; the lines marked 🔒 are encrypted. A click on a line says what it does.
The eavesdropper shows its copy of the same record, and lists everything the copies have told it so far.
Step plays twelve scenes, one record per step; the client's own work, computing the keys and checking the certificate, is a step of its own.
The buttons on each card take the story our own way; the scene picker or Restart brings it back.
Messages so far, on the right, lists the scene's records; a click on one goes back to it.

The TLS 1.3 handshake

The handshake takes one round trip: the client sends one flight, the server answers with one, and the client's Finished closes it.

ClientHello, in plain text: the versions and cipher suites the client offers, a fresh key share, and the name of the site (SNI).
ServerHello, in plain text: the server's choice and its own key share. Both sides now compute the same secret, and every record after this one is encrypted.
EncryptedExtensions: the rest of the answer, such as http/1.1 from the client's ALPN list.
Certificate: the server's certificate and the intermediate CA's.
CertificateVerify: a signature over the whole handshake, made with the private key of the certificate.
Finished: a MAC over the handshake, proving the server saw the same messages the client did.
The client's checks, then its Finished, or a fatal alert that ends the connection.

The steps of TLS Concepts are these records under other names:

TLS Concepts	The record here
Say hello to shop.example	ClientHello
Send your key share	ServerHello
Diffie-Hellman: compute the shared session key	both sides, right after ServerHello
Send your certificate	Certificate, with the intermediate's certificate
Sign this handshake	CertificateVerify
Check who answered	the client's checks
Send the password	the request, in an `application_data` record

EncryptedExtensions and the two Finished messages have no step there: they carry only the answer's details and a proof that both ends saw the same handshake.
HTTP starts after the client's Finished: the request is the same text as on the Basic auth page, inside an application_data record.
A browser keeps the connection and sends its next request with no handshake at all; curl opens a new connection every time it runs.

The key agreement: ephemeral Diffie-Hellman

The key exchange is Diffie-Hellman on the x25519 curve, with key pairs that each side makes for this connection alone: ECDHE, the name cipher lists and openssl use.

The key shares are the public halves.
One side's private half combined with the other side's public half gives the same number on both sides, and the private halves never leave their machines.
The eavesdropper copies both public halves and still cannot compute the secret.
The keys of every record from ServerHello on are derived from that secret.
Forward secrecy follows from the key pairs being fresh: the server's long-lived private key only signs, so stealing it later decrypts none of the connections recorded before.

What the eavesdropper sees

A copy of TLS 1.3 traffic tells the eavesdropper which site, which cipher, and how big each record is.

The site's name travels in plain text in the ClientHello's server_name, because the server needs it to choose a certificate before anything is encrypted.
Every encrypted record is labelled application_data, whatever it carries.
The length stays visible: an encrypted record is 17 bytes longer than its contents:
- one byte for the real type;
- a 16-byte tag that fails to verify if a single bit is changed.
The certificate is encrypted in TLS 1.3, so the eavesdropper sees a record the size of one, not which one it is.
Over plain http:// it reads everything, the Authorization header decoding to alice:secret123 included.

The client's checks of the certificate

The client checks what the server sent in a fixed order, and the first failure ends the handshake with an alert.

Check	Fails when	Alert	curl says	Chrome says
The chain ends at a root in the trust store	the certificate is self-signed, or `--cacert` left out its root	`unknown_ca` (48)	`(60) SSL certificate problem: self-signed certificate` / `unable to get local issuer certificate`	`NET::ERR_CERT_AUTHORITY_INVALID`
Today is between the dates	the certificate expired	`certificate_expired` (45)	`(60) SSL certificate problem: certificate has expired`	`NET::ERR_CERT_DATE_INVALID`
The name typed is in the SAN list	the certificate is for another name	`bad_certificate` (42)	`(60) SSL: no alternative certificate subject name matches target hostname`	`NET::ERR_CERT_COMMON_NAME_INVALID`
CertificateVerify matches the certificate's key	the sender does not hold the private key	`decrypt_error` (51)	`(35) error:0A00007B:SSL routines::bad signature`	`ERR_SSL_PROTOCOL_ERROR`

The name is matched exactly. The certificate lists shop.example and www.shop.example, so api.shop.example is refused.
curl checks the name itself, after the handshake. OpenSSL does the other checks during the handshake, so for a wrong name curl completes it, then closes the connection before sending the request.
The signature is the check that stops an impostor.
- A certificate is public, and anyone can send the real one.
- An impostor with the real certificate passes the chain, the dates and the name.
- It fails on CertificateVerify, because the private key never left the real server.

curl's -k and --cacert

The course lab serves self.crt, a certificate that signs itself, and gets past curl's refusal in two ways.

-k skips the checks of the certificate: the trust, the dates and the name.
- The connection is encrypted, to whoever answered.
- The signature is the one check it keeps: an impostor holding only the real certificate still fails.
- An impostor that makes a self-signed certificate of its own gets through.
--cacert self.crt names the root to trust for this command.
- A self-signed certificate is its own root, so curl checks everything and accepts it.
- The file replaces the system's CAs instead of adding to them, so the same flag refuses the site's real, CA-issued certificate.

The browser's Proceed (unsafe)

The browser's warning page is curl's refusal, shown as a page.

Proceed is the browser's -k, for one certificate on one site.
The browser keeps an exception, opens a new connection, and accepts that certificate from then on, with Not secure in the address bar.
The warning for a failed signature offers no Proceed at all, since the server behind it cannot be told from an impostor.

The course Authentication and Secrets runs the same handshake against nginx in its chapter TLS, with curl -v, openssl s_client and a self-signed certificate.