Computer Networks #14: SSL/TSL Handshake
We have already discussed how the transport layer protocol and application layer protocols work under the hood. But there is one very important thing we have not discussed yet:
How is all of this communication made secure?
Imagine that you open:
https://example.com
Your request travels through a network that you do not completely control. An attacker could potentially sit somewhere in between and try to read your data, modify it, or even pretend to be the real server.
This is where TLS, Transport Layer Security, comes into the picture.
TLS is the security layer that sits between the application and transport layers. It provides encryption, integrity, and authentication for the connection.
Let's understand the complete flow step by step.
Before TLS, TCP handshake happens
When the client wants to communicate with a server using HTTPS, the first thing that happens is the normal TCP 3-way handshake.
Client Server
SYN ------------------------>
<---------------- SYN + ACK
ACK ------------------------>
TCP connection established
At this point, TCP has created a reliable connection between the client and server.
But there is a problem.
TCP does not encrypt the data.
So now we need another protocol to make this connection secure.
That protocol is TLS.
Step 1: The server has a long-term key pair
Before the client even connects, the server has a public/private key pair.
Think of it like this:
Server
Private Key 🔐
Public Key 🔓
The private key is kept secret on the server.
The public key is not secret and can be shared with others.
The server also has a digital certificate issued by a Certificate Authority, or CA.
The certificate contains information such as:
Domain: example.com
Public Key: Server Public Key
Issued By: Certificate Authority
CA Signature: ...
So the certificate is basically a digital identity card for the server.
It tells the client:
"This public key belongs to example.com."
The important thing to understand is that the server's private key stays on the server. It is never sent to the client.
Step 2: Client sends ClientHello
After the TCP connection is established, the TLS handshake begins.
The client sends a message called ClientHello.
The client is basically saying:
"Here are the TLS versions I support, the cryptographic algorithms I support, and other information needed to establish a secure connection."
In TLS 1.3, the ClientHello also contains a temporary Diffie-Hellman key share that will be used later to create a shared secret.
Conceptually:
Client
|
| ClientHello
| - Supported TLS versions
| - Supported algorithms
| - Temporary public key
|
v
Server
Step 3: Server sends ServerHello
The server receives the ClientHello and chooses the parameters that both sides will use.
It sends back a ServerHello.
The server is basically saying:
"Okay, we will use these TLS settings."
In TLS 1.3, the ServerHello also contains the server's own temporary Diffie-Hellman public key share.
So now we have:
Client Server
Client Temporary Public Key --->
<--- Server Temporary Public Key
These temporary keys are not the same thing as the public key inside the certificate.
This distinction is extremely important.
We now have two different types of keys.
Long-term key pair
Server Private Key
Server Public Key
Used mainly for server authentication.
Temporary key pair
Client Temporary Private Key
Client Temporary Public Key
Server Temporary Private Key
Server Temporary Public Key
Used for Diffie-Hellman key exchange.
Step 4: Server sends its certificate
The server now sends its certificate.
Conceptually, the certificate contains:
Certificate
Domain:
example.com
Public Key:
Server Public Key
Signed By:
Trusted CA
The client now checks the certificate.
It asks questions like:
Is this certificate from a trusted CA?
Does it belong to example.com?
Is it valid?
Has it expired?
The browser or operating system has information about trusted Certificate Authorities.
So if the certificate was properly signed by a trusted CA and is valid for the requested domain, the client accepts it.
This is what prevents a random attacker from simply saying:
"I am example.com."
The attacker can create their own key pair, but they cannot normally create a trusted certificate saying that their public key belongs to example.com.
Step 5: How does the server prove it actually owns the private key?
This is where the server's long-term private key becomes important.
The server has:
Server Private Key 🔐
and its certificate contains:
Server Public Key 🔓
The server uses its private key to create a digital signature over the handshake information.
This is sent in the CertificateVerify message.
The client then uses the public key from the certificate to verify that signature. TLS 1.3 explicitly uses CertificateVerify as proof that the server possesses the private key corresponding to the public key in its certificate.
Conceptually:
Server
Private Key
|
v
Create signature
|
v
Send signature
The client does:
Certificate
|
v
Get Server Public Key
|
v
Verify signature
If the verification succeeds:
✅ The server really has the private key
✅ The certificate's public key belongs to this server
✅ This is the server identity we expected
So the certificate and private key together solve the authentication problem.
Step 6: Now comes the most interesting part: creating a shared secret
At this point, the client and server have exchanged their temporary Diffie-Hellman public keys.
Remember:
Client:
Temporary Private Key
Temporary Public Key
Server:
Temporary Private Key
Temporary Public Key
The client uses:
Client Temporary Private Key
+
Server Temporary Public Key
to calculate a shared secret.
At the same time, the server uses:
Server Temporary Private Key
+
Client Temporary Public Key
to calculate a shared secret.
Both sides arrive at the same secret.
Client Server
Client Temp Private Server Temp Private
+ +
Server Temp Public Client Temp Public
| |
v v
Shared Secret X Shared Secret X
The most important thing here is:
The shared secret itself is never sent over the network.
The attacker can see the temporary public keys, but does not have the corresponding temporary private keys.
So the attacker cannot practically calculate the same shared secret.
This is the main idea behind Diffie-Hellman key exchange. TLS 1.3 uses ephemeral Diffie-Hellman key shares for this purpose.
Step 7: TLS derives the actual traffic keys
The shared secret is not simply used directly as the encryption key.
TLS uses a key derivation function, specifically HKDF in TLS 1.3, to derive the actual secrets and traffic keys used by the connection.
Conceptually:
Shared Secret
|
v
HKDF
|
v
Traffic Keys
Now both sides have matching keys for protecting the communication.
Client Server
Traffic Key 🔑 Traffic Key 🔑
The client and server each calculated these keys independently.
Nobody transmitted the final encryption key directly.
Step 8: Actual HTTP data is now encrypted
Now the TLS handshake is complete.
The client can finally send the actual HTTP request.
For example:
GET /profile
But it does not travel across the network as plain text.
Instead:
HTTP Request
|
v
TLS Traffic Key
|
v
Encrypted TLS Data
So an attacker sitting in the middle sees encrypted data instead of the original HTTP request.
Client
|
| Encrypted Data 🔒
v
Attacker
|
| Encrypted Data 🔒
v
Server
The server can decrypt it because it independently calculated the same traffic keys during the handshake.
So how does the Man-in-the-Middle attack fail?
Let's imagine that an attacker tries to place themselves between the client and server.
They want this:
Client --------> Attacker --------> Server
The attacker could generate their own temporary key pair.
That part is easy.
The problem is authentication.
The attacker would need to convince the client:
"My public key belongs to example.com."
But the client verifies the server's certificate through the trusted CA system.
The attacker cannot simply create a certificate for example.com signed by a trusted CA.
And even if the attacker creates their own temporary Diffie-Hellman keys, they still cannot produce a valid CertificateVerify signature for example.com because they do not possess the real server's private key.
So the attack fails.
One diagram for the complete TLS flow
Here is the whole process in one place:
TCP 3-WAY HANDSHAKE
Client Server
| |
| -------- SYN ---------------------------> |
| <------- SYN + ACK --------------------- |
| -------- ACK ---------------------------> |
| |
| TCP connection established |
| |
| |
| TLS HANDSHAKE |
| |
| -------- ClientHello ------------------> |
| - TLS versions |
| - Algorithms |
| - Client temporary public key |
| |
| <-------- ServerHello ------------------ |
| - Selected parameters |
| - Server temporary public key |
| |
| <-------- Certificate ------------------ |
| - example.com |
| - Server public key |
| - CA signature |
| |
| <-------- CertificateVerify ------------ |
| Server proves it owns |
| the certificate's private key |
| |
| |
| Client and server perform |
| Diffie-Hellman calculation |
| |
| Shared Secret created on |
| both sides independently |
| |
| ↓ |
| HKDF |
| ↓ |
| Traffic Keys created |
| |
| |
| ========== SECURE CONNECTION ============ |
| |
| -------- Encrypted HTTP Request --------> |
| <-------- Encrypted HTTP Response ------- |
