We'll cap off our discussion of authentication by looking at a real-world protocol that ensures confidentiality and integrity of messages, as well as authenticating the identity of machines: the Secure Sockets Layer (SSL).
There are actually two names for this protocol. SSL was invented by Netscape, and later was standardized under the name Transport Layer Security (TLS). SSL v1 was never released, v2 was flawed, v3.0 was pre-standardization. After that came:
TLS 1.0 was broken by something called the BEAST attack in September 2011; the attack exploits a vulnerability in the way that TLS chose IVs for CBC mode. Many implementations are (finally) being updated to TLS 1.1.
Recall the Internet model of the network stack:
SSL fits between the transport and application layers, providing confidentiality and integrity on top of the guarantees already made at the transport layer for reliable message streams. Application-layer protocols then rely on those extended guarantees. For example, HTTPS is HTTP over SSL, and SMTP can run over SSL, too.
SSL manages sessions. A session is a bi-direction communication between a client and a server. The communication is optionally secured for both confidentiality and integrity against a Dolev-Yao attacker. Sessions are logical, in that there can be many sessions between any two physical hosts, and each host could be either client or server in any given session.
Each message sent during a session is called a record. SSL consists of two protocols: the record protocol, and the handshake protocol.
The purpose of the record protocol is to exchange records between the client and server.
The record protocol can run in an unsecured mode, with no protection of confidentiality or integrity. But normally, the record protocol uses a block cipher (and mode) to protect confidentiality, and a MAC to protect integrity. Those symmetric crypto algorithms require several keys and other values, which collectively are called the connection state:
cmk: client HMAC key
smk: server HMAC key
cek: client (symmetric) encryption key
sek: server (symmetric) encryption key
civ: client IV (for block mode)
siv: server IV (for block mode)
cseq: client sequence number
sseq: server sequence number
All the state is doubled (half for the client, half for the server), because communication is directional: different keys (etc.) are used when the client sends to the server vs. when the server sends to the client. One purpose of that is to prevent reflection attacks, in which an adversary attempts to reflect a message back to a principal thus changing whether the principal is acting as the client or the server. The sequence numbers prevent the adversary from reordering messages. Note that keys 1-4 are shared: both the client and server know them, even though name is (e.g.) the "client" encryption key.
Although we've specified a block cipher mode here, SSL can also use stream ciphers, which is an alterative way of extending a fixed-length block cipher to an arbitrary length message. Also, the most recent standard (TLS 1.2) allows use of authenticated encryption block ciphers, which do not use HMACs. But to simplify presentation, we assume the use of an HMAC and a block cipher mode.
The protocol used to send messages from the client to the server in the record protocol is essentially just an authenticated encryption using Mac-then-Encrypt.
For client `C` to send record `r` to server `S`: 1. C: t = MAC(r, cseq; cmk) c = Enc(r, t; cek) cseq++; if overflow, renegotiate keys civ = rand() // missing from TLS 1.0 2. C->S: c
And the protocol used from the server to the client is essentially the same:
For `S` to send `r` to `C`: 1. S: t = MAC(r, sseq; smk) c = Enc(r, t; sek) sseq++; if overflow, renegotiate keys siv = rand() // missing from TLS 1.0 2. S->C: c
So how does SSL establish those shared keys? That's the job of the handshake protocol.
The purpose of the handshake protocol is to establish sessions, setting up the connection state by generating and sharing keys, negotiating what cryptographic algorithms to use, and optionally exchanging digital certificates for authentication. There are three choices to make for cryptographic algorithms:
Collectively, this triple of algorithms is called the cipher suite. Here are a few examples of cipher suites:
null, null, null(i.e., completely unsecured mode for SSL)
DH_anon, DES/CBC, SHA-1(i.e., exchange keys using anonymous Diffie-Hellman which actually does no authentication of the principals' identities, use DES in CBC mode for encryption, and use SHA-1 for hashing; note that the latter two are deprecated, but many implementations still enable them)
RSA, AES128/CBC, SHA-256(i.e., exchange keys using RSA, use AES-128 in CBC mode for encryption, and use SHA-256 for hashing)
The wise user of a crypto library will always ensure that the cipher suite being chosen by the library does not include old, deprecated algorithms. You might be surprised what some libraries use as defaults. Indeed, a number of attacks against SSL work by causing roll-back to deprecated algorithms.
We'll assume the use of RSA to exchange keys to simplify the rest of the presentation.
---start using record protocol in unsecured mode 1. C->S: Suites_C 2. S->C: Suite_S, CA<<S>> // cert might be a chain 3. C: generate random premaster secret (PS) // 48 bytes ePS = Enc(PS; K_S) 4. C->S: ePS 5. S: PS = Dec(ePS; k_S) 6. C and S: MS = H(PS) from MS, derive connection state by simply splitting MS into bit strings ---switch record protocol to secured mode
Problem: the client got to pick all the randomness in this protocol. Maybe the client does a bad job of choosing randomness. So the server shouldn't necessarily trust the resulting keys.
Problem: client and server both want to make sure this is a fresh run of protocol that can't be confused with previous runs.
Solution: both client and server contribute a nonce.
The new part of this attempt is the nonces
N_S, which we surround by double asterisks below just to call them out:
---start using record protocol in unsecured mode 1. C->S: Suites_C, **N_C** 2. S->C: Suite_S, CA<<S>>, **N_S** 3. C: generate random PS ePS = Enc(PS; K_S) 4. C->S: ePS 5. S: PS = Dec(ePS; k_S) 6. C and S: MS = H(PS, **N_C**, **N_S**) ---switch record protocol to secured mode
Problem: what if the adversary changes some messages?
S might end up computing different keys. Or they might end up choosing a ciphersuite that is weaker than they wanted. So they need to verify, before going on, that they have the same keys, and those keys are from the current run of protocol (to prevent replays by the adversary). They do that by authenticating all the messages sent during the handshake, proceeding as follows using the record layer in secure mode:
7. C: cRcvMsgs = H(Suites_C, Suite_S, CA<<S>>, N_C, N_S, ePS) 8. C->S: cRcvMsgs // i.e., AuthEnc(cVerMsgs; cek; cmk) 9. S: verify that all messages match what S sent and received 10. S: sRcvMsgs = H(Suites_C, Suite_S, CA<<S>>, N_C, N_S, ePS, cRcvMsgs) 11. S->C: sRcvMsgs // i.e., AuthEnc(sVerMsgs; sek; smk) 12. C: verify that all messages match what C sent and received
The handshake protocol we just developed does unilateral authentication, meaning that only one of the principals is authenticated to the other. Here, the server authenticates itself to the client, but the client does not reciprocate.
With mutual authentication, the client would also authenticate itself to the server. This requires the client to present a certificate (chain) to the server, which it would include in step 4 along with the encrypted premaster secret. To prove that the client knows the private signing key corresponding to the public verification key in that certificate, the client would also sign all the messages up to that step and include that signature, which the server would check.
SSL offers another authentication option called anonymous key exchange, which is based on a cryptographic protocol called Diffie-Hellman (after its two inventors). With this option, neither principal is authenticated to the other. So it is quite vulnerable to man-in-the-middle attacks.
Why not read the TLS specification! You can find it at http://tools.ietf.org/html/rfc5246.