TLS Handshake

This page or section is a work in progress and may thus be incomplete. Its content may be changed in the near future.

Any TLS communication starts with a TLS handshake, which establishes what protocol will be used. We will focus on this page about what happens when the TLS_DHE_RSA_WITH_AES_128_CBC_SHA cipher suite is used (see SSL/TLS for more information about what this means)

Handshake Overview

Most packets during the communication are of type Handshake (0x16) and are followed by a Handshake packet header:

typedef struct __attribute__((packed)) {
	uint8_t content_type;  // 0x16
	uint16_t version;
	uint16_t length;
} TLSRecord;

This header may be followed by another TLS header, such as a TLS Handshake header. Like for a TCP connection, a TLS connection starts with a handshake between the client and the server:

• The client sends a Client Hello message, including a list of 32-byte list of random data and the list of its supported cipher suites. In our example we only send one supported cipher suite (code 0x0033)
• The server responds with a Server Hello message, telling the client what cipher suite is going to be used as well as its own 32-byte list of random data
• The server sends its certificates. These are used by the client to verify that it is actually talking to the site it thinks it is talking to, as opposed to a malicious site
• The server sends a Server Key Exchange message, initiating the key exchange and signing it with its public key
• The server sends a Server Hello Done message, indicating it is waiting for the client
• The client sends a Client Key Exchange message, containing its part of the key exchange transaction
• The client sends a Change Cipher Spec message
• The client sends a Encrypted Handshake Message
• The server sends a Change Cipher Spec
• The server sends a Encrypted Handshake Message
• The client and the server can communicate by exchanging encrypted Application Data messages

The Change Cipher Spec message tells the other party its is OK with the terms of the handshake.

The Encrypted Handshake messages are the first ones to be sent encrypted. They contain a hash of the initial handshake messages and are here to ensure these were not tampered with.

Any subsequent communication is of type Application Data and encrypted.

Client Hello Message

The Client Hello message initiates the TLS handshake. It is composed of a specific header, followed by some (optional) extensions, followed by some optional padding. If some servers such as wikipedia.org are quite forgiving in the types of Client Hello messages they accept, others such as Google require the Client Hello message to be exactly 512 bytes (excluding the TLS Record header) and declare a server_name extension

Here is an example of a Client Hello message:

0000   01 00 01 fc 03 03 57 16 ea ce ec 93 89 5c 4a 18
0010   d3 1c 5f 37 9b b3 05 b4 32 08 29 39 b8 3e e0 9f
0020   9a 96 ba be 0a 40 00 00 02 00 33 01 00 01 d1 ff
0030   01 00 01 00 00 00 00 16 00 14 00 00 11 77 77 77
0040   2e 77 69 6b 69 70 65 64 69 61 2e 6f 72 67 00 0d
0050   00 12 00 10 06 01 06 03 05 01 05 03 04 01 04 03
0060   02 01 02 03 00 0b 00 02 01 00 00 0a 00 06 00 04
0070   00 17 00 18 00 15 01 88 00 00 00 00 00 00 00 00
0080   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
...
01e0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
01f0   00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

• 0x01: Client Hello handshake type
• 0x0001FC: size=508 (512-4)
• 0x0303: TLS 1.2
• 0x5716...0A40: 32-bytes random client data
• 0x00: session ID length=0 (if it was not null, it would be followed by the session ID)
• 0x0002: Cipher Suites Length=2 (1 cipher suite supported)
  • 0x0033: TLS_DHE_RSA_WITH_AES_128_CBC_SHA cipher suite code
• 0x01: number of compression methods=1
  • 0x00: compression method=null
• 0x01D1: extensions length=465
• 0xFF01: extension type=renegotiation_info
  • 0x0001: length=1
  • 0x00: renegotiation info extension length=0
• 0x0000: extension type=server_name (the domain name we are trying to contact)
  • 0x0016: length=22
  • 0x0014: server name list length=20
  • 0x00: server name type=host_name
  • 0x0011: server name length=17
  • 0x7777772E...7267: "www.wikipedia.org"
• 0x000D: extension type=signature_algorithms (the signature algorithms supported)
  • 0x0012: length=18
  • 0x0010: signature hash algorithms length=16
    • 0x0601: SHA512 + RSA
    • 0x0603: SHA512 + ECDSA
    • 0x0501: SHA384 + RSA
    • 0x0503: SHA384 + ECDSA
    • 0x0401: SHA256 + RSA
    • 0x0403: SHA256 + ECDSA
    • 0x0201: SHA1 + RSA
    • 0x0203: SHA1 + ECDSA
• 0x000B: extension type=ec_point_formats (only used if you use elliptic curve cryptography)
  • 0x0002: length=2
  • 0x02: EC point formats length=1
  • 0x00: EC point format=uncompressed
• 0x000A: extension type=elliptic_curves (Elliptic Curve types supported - only used if you use elliptic curve cryptography)
  • 0x0006: length=6
  • 0x0004: elliptic curve length=4 (2 curves)
  • 0x0017: secp256r1 elliptic curve
  • 0x0018: secp384r1 elliptic curve
• 0x0015: extension type=padding (we fill the rest of the 512 bytes with zeros)
  • 0x0188: length=392
  • 0x0000...0000: 392 bytes of padding data

Server Hello Message

The Server Hello message indicates which cipher suite is going to be used and provides some server random data that will be used later on.

Here is a sample Server Hello message:

0000   02 00 00 4d 03 03 c8 ca 5c 5f 83 79 eb 8f 8a 16
0010   86 c2 07 d7 42 c7 ee b9 dc 71 b7 f1 71 9f eb 51
0020   66 24 b4 1e 4f 6c 20 e4 02 80 10 1d ea 7f aa ee
0030   5d 4f ac 53 49 29 25 ec 29 a8 b7 23 fa ef 24 d4
0040   47 2e 90 7b 99 36 2b 00 33 00 00 05 ff 01 00 01
0050   00

• 0x02: Handshake type=Server Hello
• 0x00004D: length=77
• 0x0303: TLS version 1.2
• 0xC8CA...4F6C: 32-bytes server random data
• 0x20: session ID length=32
• 0xE402...362B: session ID (can be used in a future TLS connection to avoid going through the handshake again)
• 0x0033: cipher suite used=TLS_DHE_RSA_WITH_AES_128_CBC_SHA
• 0x00: compression method used=null
• 0x0005: extensions length=5
• 0xFF01: extension type=renegotiation_info
  • 0x0001: length=1
  • 0x00: renegotiation info extension length=0

Certificate Message

The server then sends a Certificate message containing its SSL Certificate chain. The first certificate is the server's SSL certificate. The next certificate is the certificate from a Certificate Authority (CA) which signed the first certificate. The next certificate signs the previous certificate, and so on. The last certificate in the chain should belong to a root CA and is self-signed (each TLS client should have a list of all the root CAs)

Here is how the Certificate Message is encoded:

• 0x0B: handshake type=Certificate
• 0x000C58: length=3160
• 0x000C55: certificates length=3157
  • 0x0007E2: certificate #1 Length=2018
    • 0x3082...C0F3: first certificate (ASN.1 encoded)
  • 0x00046D: certificate #2 length=1133
    • 0x3080...4998: second certificate (ASN.1 encoded)

Key Exchange

TLS encryption is performed using symmetric encryption. The client and server thus need to agree on a secret key. This is done in the key exchange protocol.

In our example, TLS is using the DHE/RSA algorithms: the Diffie-Hellman Ephemeral protocol is used to come up with the secret key, and the server is using the RSA protocol to sign the numbers it sends to the client (the signature is linked to its SSL certificate) to ensure that a third party cannot inject a malicious number. The upside of DHE is that it is using a temporary key that will be discarded afterwards. Key exchange protocols such as DH or RSA are using numbers from the SSL certificate. As a result, a leak of the server's private key (for example through Heartbleed) means that a previously recorded SSL/TLS encryption can be decrypted. Ephemeral key exchange protocols such as DHE or ECDHE offer so-called forward secrecy and are safe even if the server's private key is later compromised.

Diffie-Hellman Ephemeral works as follows:

• The server comes up with a secret number y, with a number g and a modulo p (p typically being a 1024 bit integer) and sends (p, g, pubKey=g^y mod p) to the client in its "Server Key Exchange" message. It also sends a signature of the Diffie-Hellman parameters (see SSL Certificate section)
• The client comes up with a secret number x and sends pubKey=g^x mod p to the server in its "Client Key Exchange" message
• The client and server derive a common key premaster_secret = (g^x)^y mod p = (g^y)^x mod p = g^xy mod p. If p is large enough, it is extremely hard for anyone knowing only g^x and g^y (which were transmitted in clear) to find that key.

Because computing g^xy mod p using 1024-bytes integers can be tedious in most programming languages, if security is not a concern, one way to avoid this is to use x=1. This way, premaster_secret is just g^y mod p, a value directly sent by the server. The security in such a case is of course compromised.

premaster_key is however only a first step. Both client and server uses the PRF function to come up with a 48-byte master secret. The PRF function is used once again to generate a 104-bytes series of data which will represent all the secret keys used in the conversation (the length may differ depending on the cipher suite used):

# g_y, g and p are provided in the Server Key Exchange message
# The client determines x
premaster_secret = pow(g_y, x, p)
 
# client_random and sever_random are the 32-bytes random data from the Client Hello and Server Hello messages
master_secret = PRF(premaster_secret, "master secret", client_random + server_random, 48)
keys = PRF(master_secret, "key expansion", server_random + client_random, 104)
 
# The MAC keys are 20 bytes because we are using HMAC+SHA1
client_write_MAC_key = keys[0:20]
server_write_MAC_key = keys[20:40]
# The client and server keys are 16 bytes because we are using AES 128-bit aka a 128 bit = 16 bytes key
client_write_key = keys[40:56]
server_write_key = keys[56:72]
# The IVs are always 16 bytes because AES encrypts blocks of 16 bytes
client_write_IV = keys[72:88]
server_write_IV = keys[88:104]

Note how different secret keys are used for the client and for the server, as well as for encryption and to compute the MAC.

Server Key Exchange Message

The server then sends its part of the key exchange. In the following case it will send the Diffie-Hellman parameters p, g and pubKey=g^y

• 0x0C: handshake type=server key exchange
• 0x00030B: length=779
• 0x0100: p length=256
  • ..... : 256-bytes p (1024-bit integer)
• 0x0001: g length=1
  • 0x02: g = 2
• 0x0100: pubKey length
  • ... : 256-bytes pubKey
• 0x0601: signature hash algorithm used to sign the Diffie-Hellman parameters=SHA512+RSA
• 0x0100: signature length=256
  • ... : 256-bytes signature

Server Hello Done Message

This message indicates that the server is done and is awaiting for client input.

typedef struct __attribute__((packed)) {
	uint8_t handshake_type;  // 0x0E
	uint8_t length[3];  // 0x000000
} TLSServerHelloDone;

Client Key Exchange Message

The client then sends its key exchange parameters: pubKey=g^x

• 0x10: handshake type=client key exchange
• 0x000102: length=258
• 0x0100: pubKey length=256
• ...: 256-bytes pubKey

Change Cipher Spec Message

The client sends the Change Cipher Spec message to indicate it has completed its part of the handshake. The next message the server will expect is the Encrypted Handshake Message.

The whole message (including the TLS Record header) is 6 bytes long:

typedef struct __attribute__((packed)) {
	uint8_t content_type;  // 0x14
        uint16_t version; // 0x0303 for TLS 1.2
	uint8_t length;  // 0x01
        uint8_t content;  // 0x01
} TLSChangeCipherSpec;

Another Key Exchange: Elliptical Curve Diffie Hellman Ephemeral

If Diffie-Hellman is a very powerful algorithm, it requires very large numbers to be considered secure (1024-bit at minimum). A variant is Elliptical Curve Diffie-Hellman, which is much harder to break even with 256-bit numbers. Numerous TLS cipher suites now rely on the ECDHE_RSA key exchange instead of DHE_RSA, like in the TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA cipher suite.

Elliptic Curve Diffie-Hellman works as follows: consider a point G = (x, y) on a curve y² = x³ + a.x + b mod p. Both parties come up with secret numbers secret1 and secret2, and will send each other G.secret1 and G.secret2 (G.secret1 means multiplying the point G to secret1 using Elliptic Curve point multiplication). The shared secret key is G.secret1.secret2.

TLS can use the ECDHE key exchange to come up with an ephemeral shared secret key the following way:

• The server indicates in the Server Key Exchange message what type of curve is going to be used (secp256r1 is a very common one). This tells what parameters a, b, p and G to use (see [1] to see the domain parameters for each curve)
• The server comes up with a random 256-bit number (or whatever the curve says) server_secret and sends pubKey = G*server_secret in the Server Key Exchange message. pubKey is sent as a 65-bytes block composed of the concatenation 0x04 | Gx | Gy (both numbers being 32-bytes long)
• The client comes up with a random 256-bit number client_secret and sends pubKey = G*client_secret in the Client Key Exchange message. pubKey is sent in the same format as the server's
• Both parties will derive premaster_secret by computing server_pubKey * client_secret = client_pubKey * server_secret = G * client_secret * server_secret. The x coordinate of this result is the premaster_secret
• Once premaster_secret is determined, the rest of the computation works the same regardless of the key exchange protocol used

Regarding how to compute elliptic curve point multiplication, Wikipedia offers more details. Note that, because we are only dealing with integers, you should use modular multiplicative inverse instead of divisions.

If you want to test Elliptic Curves in Python, TinyEC is a very useful package (along with the source code in pure Python):

import tinyec.ec as ec
import tinyec.registry as reg
 
# Get the domain parameters for the named curve specified in the Server Key Exchange message
curve = reg.get_curve("secp256r1")
 
# Comes up with a random 256-bit (32 bytes) client_secret
# curve.g is a point on the elliptic curve, defined by the domain parameters
# We multiply it with client_secret to obtain the public key
client_pubKey = curve.g * client_secret
# Retrieved from the Server Key Exchange message
server_pubKey = ...
 
premaster_secret = (server_pubKey * client_secret).x

Another Key Exchange: RSA

The RSA key exchange is by today's standard an old key exchange protocol and does not provide forward secrecy. It is however simpler to implement than ECDHE and still supported by multiple sites (for example, Google supports the RSA key exchange but not the simple Diffie-Hellman key exchange).

With the RSA key exchange, the server does not send any "Server Key Exchange" message. Instead, the client decides the premaster_secret, which is a 48-bytes string composed of a two-bytes TLS version (0x0303 for TLS 1.2) followed by 46 random bytes. It then encrypts that premaster_secret using the PKCS #1 protocol (aka RSA encryption version 1.5) as well as the key from the Web site's certificate as the public key. Because the public key is always the same, a leak of the Web site's private key would allow to decrypt previously recorded TLS conversations. This is why the RSA key exchange protocol offers no forward secrecy.

import os
from Crypto.Cipher import PKCS1_v1_5
from Crypto.PublicKey import RSA
 
# Come up with a random premaster_secret
# It is recommended to use a random generator from a crypto library if possible
premaster_secret = '\x03\x03' + os.urandom(46)
 
# Retrieve the pubKey from the first certificate. This pubKey value (65-bytes for a 1024-bit public key) is stored in the ASN.1 format
key = RSA.importKey(pubKey)
 
# Encrypt the premaster_secret
cipher = PKCS1_v1_5.new(key)
encrypted_premaster_secret = cipher.encrypt(premaster_secret)

Without the Crypto.Cipher.PKCS1_v1_5 package:

import os
 
# Come up with a random premaster_secret
premaster_secret = '\x03\x03' + os.urandom(46)
 
# Retrieve the pubKey from the first certificate (parse_ASN1 is a fictional function)
RSA_n, RSA_e = parse_ASN1(pubKey)
 
# Encrypt the premaster_secret
premaster_secret = '\x00\x02' + '\x42' * (256 - 3 - len(premaster_secret)) + '\x00' + premaster_secret
encrypted_premaster_secret = pow(to_int(premaster_secret), RSA_e, RSA_n)

SSL Certificate (optional)

In order to prevent a Man-In-The-Middle attack (MITM), the server will sign the Diffie-Hellman parameters it sent to the client. Because the client may have never contacted the server before (and thus cannot securely obtain its public key), the client and server rely on a trusted third party known as a Certificate Authority (CA).

In order to verify the signature using the RSA algorithm, the client need to do the following:

• Retrieve the Certificate message sent by the server, which contains one or more certificates (look at a such a packet in Wireshark)
• Verify that the first certificate's RDN sequence (signedCertificate / subject:rdnSequence / rdnSequence) contains the Web site the client is trying to contact
• Get the RSA e and n values from the first certificate's public key (signedCertificate / subjectPublicKeyInfo / subjectPublicKey). Those parameters are encoded using the ASN.1 format (as a verification, e is very often 65537, or 0x10001)
• Compute the hash of the whole DH parameters (as sent by the server) preceded with the client and server random data. The certificate indicates what type of hash to use (signedCertificate / subjectPublicKeyInfo / algorithm):
• Compute signature^e mod n, convert it to a string and take the last 20 bytes (or more, depending on the hash function being used)
• Both computations should be the same
• Because this certificate is probably generated by an intermediate CA, the client needs to verify that certificate
• Compute the hash of the whole signedCertificate section and repeat the operation using the next certificate
• Follow the certificate chain up to the end. The last certificate should belong to a root CA (any TLS implementation should contain a list of the root CAs and their public key) and is self-signed

Encrypted Handshake Message

The TLS handshake is concluded with the two parties sending a hash of the complete handshake exchange, in order to ensure that a middleman did not try to conduct a downgrade attack.

If your TLS client technically does not have to verify the Encrypted Handshake Message sent by the server, it needs to send a valid Encrypted Handshake Message of its own, otherwise the server will abort the TLS session.

Here is what the client needs to do to create :

• Compute a SHA256 hash of a concatenation of all the handshake communications (or SHA384 if the PRF is based on SHA384). This means the Client Hello, Server Hello, Certificate, Server Key Exchange, Server Hello Done and Client Key Exchange messages. Note that you should concatenate only the handshake part of each TLS message (i.e. strip the first 5 bytes belonging to the TLS Record header)
• Compute PRF(master_secret, "client finished", hash, 12) which will generate a 12-bytes hash
• Append the following header which indicates the hash is 12 bytes: 0x14 0x00 0x00 0x0C
• Encrypt the 0x14 0x00 0x00 0x0C | [12-bytes hash] (see the Encrypting / Decrypting data section). This will generate a 64-bytes ciphertext using AES-CBC and 40 bytes with AES-GCM
• Send this ciphertext wrapped in a TLS Record

The server will use a similar algorithm, with two notable differences:

• It needs to compute a hash of the same handshake communications as the client as well as the decrypted "Encrypted Handshake Message" message sent by the client (i.e. the 16-bytes hash starting with 0x1400000C)
• It will call PRF(master_secret, "server finished", hash, 12)