Applied Cryptography

If you think cryptography is the answer to your problem, then you don't know what your problem is. —Peter G. Neumann

Crypto is an important building block for security. But there's much more to security than just crypto.

This material is dangerous. You won't know enough about crypto when we're done, but you'll go off and use it anyway. Be very suspicious of yourself. Take further courses in cryptography if you really want to play in this space.

There are two branches of crypto: modern and applied.

We're exclusively covering applied crypto.

Part 1: Confidentiality

Threat: An attacker who controls the communication network. This attacker can arbitrarily read, modify, and delete messages. Think of this communication model as one in which messages are always sent to the attacker, never to the intended recipient. The attacker can then forward the message along if he chooses, redirect the message, save it for later replay, etc. This kind of threat is called a Dolev–Yao attacker.

Harm: Messages containing secret information could be disclosed to the adversary, thus violating confidentiality.

Vulnerability: The communication channel between sender and receiver can be read by untrusted principals.

Countermeasure: Encryption.

Shared-key Encryption:

  1. Alice: c = Enc(m; k)
  2. Alice → Bob: c
  3. Bob: m = Dec(c; k)

(The format we use above is a protocol narration: each step is numbered and is either a computation or a message. We identify the principal(s) involved at each step by writing their names followed by a colon.)

Enc is the encryption algorithm; Dec is decryption. Alice and Bob must somehow share a key k that has previously been generated:

  1. k = Gen(len) // len is length of key
  2. ...

Together, (Gen,Enc,Dec) constitute an encryption scheme or cryptosystem. Well known examples of encryption schemes include AES (which uses shared keys) and RSA (which does not).

What makes an encryption scheme secure?

There is a provably perfectly secure encryption scheme called the one-time pad. Gen must generate a uniformly random sequence of bits of the same length as the message to be encrypted. Enc simply xors those random bits with the message, and Dec is identical to Enc. There are practical problems to deploying this scheme: (i) the keys must be really long (as long as the messages), (ii) you may never re-use a key (because doing so would reveal relationships between messages: (m1 ⊕ k) ⊕ (m2 ⊕ k) = m1 ⊕ m2), (iii) hence distributing the keys is difficult. Practical schemes instead rely on one short key that can be reused for many messages.

Block Ciphers

Efficient encryption schemes usually operate on fixed-size messages called blocks. Such schemes are called block ciphers.

Well-known examples:

When a block cipher has multiple key lengths available, we indicate the particular length being used by appending it to the name of the cipher. AES-192, for example, means AES with 192 bit keys.

Breaking a Cryptosystem

Assume the attacker doesn't know the key under which a ciphertext was encrypted. A brute force or exhaustive search means trying every possible key to decrypt a ciphertext (e.g., for AES-128, 2^128 tries). A break of a cryptosystem is an attack that succeeds in fewer steps than brute force. (e.g., only 2^99.5 tries for AES-256, which is what one theoretical, impractical attack already achieves).

If 2^X is the number of tries necessary to find the plaintext, then X is the security level of an encryption scheme. In the best case, the security level equals the key length. In practice, the security level goes down as attacks are discovered. E.g., 3DES-168 has a known attack that requires only 2^112 tries, reducing its security level from 168 to 112. Currently no practical attacks are known for AES, so—for now—its security level remains at the key length.

Recommended Key Lengths

Various entities publish recommendations for security levels based on known attacks, hardware capabilities, and predicted advances. This website summarizes NIST's recommendations, as well as others: http://www.keylength.com/en/4/.

Block Cipher Modes

If block ciphers work only on fixed length blocks, how can we send longer messages than the block length? A block cipher mode is an algorithm that uses a fixed-length block cipher to send an arbitrary-length message.

Strawman idea: chunk message into blocks; encrypt each block individually. Ciphertext block number i, written c_i, is thus Enc(m_i; k), where m_i is plaintext block number i. This algorithm is called electronic codebook mode (ECB).

ECB:
c_i = Enc(m_i; k)

ECB is a BAD IDEA that unfortunately gets invented over and over again, especially by students of crypto. Why is it bad? Because any two blocks that are same in plaintext will the be same in ciphertext. (Wikipedia has a nice graphical illustration of how ECB fails to provide confidentiality.) Do not use ECB. Unfortunately, it is still the default in Java, but you don't have to settle for the default.

One of best-known, good block cipher modes is cipher block chaining (CBC). With it, every ciphertext block depends on all previous ciphertext blocks, which avoids repetition problems like we observed with ECB.

CBC:
c_i = Enc(m_i XOR c_{i-1}; k)

If the first plaintext block is m_1, what is c_0? It can't be the encryption of a plaintext block; it has to be somehow invented from scratch for each new encryption. Block c_0 is, therefore, called the initialization vector (IV). It must be unpredictable to attackers for CBC to be secure. The best practice is to choose a new IV randomly for each (multi-block) message. The IV is sent in the clear, without encryption, because there is no meaningful information in it.

Another good block cipher mode is counter mode (CTR):

CTR:
k_i = Enc(n, i; k)
c_i = m_i XOR k_i

CTR uses Enc to encrypt a nonce n and a counter i in the same plaintext block. Observe the notation we use for that: commas between the parts of the plaintext that we're combining into one message, and a semi-colon before the key. In an implementation, we could use bit concatenation to combine message parts. Like the IV in CBC, nonce n should be randomly chosen for each new message (but stays the same for each block in the stream for a given message), and can be sent in the clear as ciphertext block c0.

An advantage of CTR over CBC is that each block in CTR can be computed in parallel, whereas CBC must process the blocks sequentially.

Nonces

Both n and the IV in the modes above are examples of a nonce: a number used once. Nonces show up a lot in crypto. A nonce must always be

Nonces can come from several sources:

Generating Random Numbers

How can software generate good, cryptographically strong random numbers? Frankly, without hardware support, it's a black art. In Java, use java.security.SecureRandom. Do not use java.util.Random, and do not use Math.random(), which itself uses java.util.Random. These latter two are predictable.

Padding

When using a block cipher mode, what should the mode do if the last block of plaintext message isn't full? Fill with 0's? No: there's no way to unambiguously remove the padding.

PKCS5 padding: Suppose B is the number of bytes that need to be added to the final plaintext block to fill it out completely. Then pad with B copies of the byte representing integer B. In the worst case, when the plaintext block is already filled, this requires adding one extra block to the message.

Public-Key Cryptography

There's a big problem with the encryption schemes we've examined so far: the shared keys have to be distributed. For each pair of principals who want to communicate, a key needs to be shared. If there are n principals, that's O(n^2) keys. That sharing costs time and money. It used to be a big problem for militaries; they had to physically distribute code books containing the keys throughout the world.

This problem motivated the invention of another kind of encryption scheme: asymmetric or public key cryptography. RSA is the most famous example. The name "asymmetric" comes from the fact that different keys are used for encryption vs. decryption. In symmetric schemes like AES, the same key is used both for encryption and decryption.

In a public-key cryptosystem, every principal has its own key pair, comprising a

N.B. Our usually fastidious terminology breaks down here. Some people call symmetric schemes "secret-key schemes", even though in both symmetric and asymmetric schemes there is a key that is kept secret. And "private" key here doesn't necessarily mean that the key is personally-identifying information.

With public-key schemes, key distribution becomes much easier. We need only to publish a "phonebook" of public keys, which contains just O(n) keys. Thus we reduce from a quadratic problem to a linear problem.

Public-key Encryption:

  1. Bob: (K_B, k_B) = Gen(len)
  2. Alice: c = Enc(m; K_B)
  3. Alice → Bob: c
  4. Bob: m = Dec(c; k_B)

Note how we use upper-case K for public keys and lower-case k for private keys.

Big Integers

Asymmetric encryption schemes are usually implemented in terms of really, really big integers—not the byte arrays that symmetric schemes use. The integers used for asymmetric encryption are far too big to fit in a standard int data type. In Java, the implementation is in terms of BigInteger. However, the Java crypto library interfaces conveniently let you pass in byte arrays and handle the conversion for you.

This use of big integers might seem like a minor implementation detail, but it's important. The lesser reason it's important is that the maximum size value you can encrypt is always bounded by the key size. Try to pass in too much plaintext, and Java will throw an exception.

The bigger reason it's important is that computation on big integers is much, much slower than computation on byte arrays. How much? In Java, anywhere from one to three orders of magnitude, based on simple experiments we did comparing AES to RSA in a past semester of this course.

Padding for Asymmetric Encryption

Since asymmetric schemes use big integers, not byte arrays, padding works differently than for symmetric encryption. With RSA, the common practice is to use a padding function called OAEP: optimal asymmetric encryption padding. OAEP actually does much more than just padding, despite its name. It even takes extra precautions to improve the security of plain RSA encryption.

Block Modes for Asymmetric Encryption

Since asymmetric encryption limits the maximum size of the plaintext, you might think that we should use block modes to encrypt arbitrary-length messages. In fact, this can be done. You could use CBC or CTR. You still should not use ECB, for the same reason as before.

In practice, though, block modes don't get used with asymmetric encryption, because encrypting many blocks with an asymmetric scheme would be really slow. Instead, the typical practice is to use a combination of both asymmetric and symmetric encryption, as discussed next.

Hybrid Encryption

To efficiently encrypt a long message using public-key crypto, we use a mash-up of asymmetric and symmetric encryption called hybrid encryption.

Hybrid encryption uses a symmetric encryption scheme (Gen_S, Enc_S, Dec_S) and an asymmetric scheme (Gen_A, Enc_A, Dec_A), as well as a block cipher mode if necessary.

Hybrid Encryption:

  1. Bob: (K_B, k_B) = Gen_A(len_A)
  2. Alice:
    k_s = Gen_S(len_S)
    c1 = Enc_A(k_s; K_B)
    c2 = Enc_S(m; k_s) // using a block cipher mode
  3. Alice → Bob: c1, c2
  4. Bob:
    k_s = Dec_A(c1; k_B)
    m = Dec_S(c2; k_s)

Key k_s is an example of a session key: a key that is for a limited time then discarded. If the session key is later compromised, only those messages it protected are vulnerable—unlike if a long-term symmetric key were used. The session key in hybrid encryption is valid only for one encryption from Alice to Bob; it shouldn't be reused for future encryptions from Alice to Bob, and Bob shouldn't use it to encrypt messages to Alice.

There's still one big problem we haven't solved: how can we distribute the "phonebook" containing everyone's public key? We'll delay discussion of that until we get to authentication of machines later in the course.

Part 2: Integrity

Threat: A Dolev–Yao attacker.

Harm: The information contained in messages could be modified, thus violating integrity.

Harm: The purported sender of a message could be changed, thus violating integrity.

Vulnerability: Messages sent on the communication channel between the sender and receiver can be modified by untrusted principals.

Countermeasure: MACs and digital signatures.

NOT a Countermeasure: Encryption does not, in general, protect integrity. Encryption protects only confidentiality!

The usual mistake is to reason as follows: "The message is encrypted. If attacker changes the ciphertext, it will decrypt to nonsense. That nonsense can be detected." But that reasoning is not valid. For example:

There are some block modes designed in last decade to protect both confidentiality and integrity. But these aren't yet widely supported, including in Java.

Encryption does not protect integrity.

Like encryption, there are symmetric and asymmetric algorithms for protecting integrity. The symmetric version is called a message authentication codes (MAC). The asymmetric version is called a digital signature. Both use another primitive, hash functions, which we'll cover first.

Cryptographic Hash Functions

A cryptographic hash function, also called a message digest, takes an arbitrary size input m and produces a fixed length output H(m). The output length is typically 128–1024 bits.

The goal of a cryptographic hash is to produce a compact representation of an original object. That representation should behave much like a fingerprint:

Likewise, cryptographic hash functions must be collision resistant and one way. What makes a hash function secure? It should behave like a random function.

Cryptographic hash functions are not the same as the ordinary hash functions that are used to implement hash tables, even though both compress their inputs. Collision resistance and one way-ness are not required of ordinary hash functions.

The security level of a hash function is, in the absence of any clever attacks, half the function's output length. E.g., if the output length is 256 bits, then the security level is at most 128 bits. Why? There's a generic attack that works on all hash functions that halves the security level. It's called the birthday attack.

MD5 and SHA-1 used to be the most commonly used hash functions. But:

SHA-2, released by the NSA in 2001. is actually a whole family of algorithms, SHA-{224,256,384,512}. The name indicates the output size in bits. Each should have security level equal to its output size halved. But these are based on similar ideas to SHA-1, so there's concern that they might one day turn out to be vulnerable to similar attacks.

Next will be SHA-3. NIST held a public competition for the new algorithm. There were five finalists, all based on different ideas than SHA-1 and SHA-2, and all developed openly and peer reviewed. The winner was announced in October 2012; the name of the winning algorithm is Keccak. It is in the process of being standardized. The output size can be 224, 256, 384, or 512 bits; or a variable-length output can be produced using a variant called SHAKE.

Message Authentication Codes (MACs)

A message authentication code is an algorithm for detecting modification of messages based on a shared key.

MAC:

  1. k = Gen(len) // A and B somehow share key k
  2. A: t = MAC(m; k) // t is called the "tag"
  3. A→B: m, t
  4. B: verify t = MAC(m; k)

The length of input m to MAC may be arbitrary. The output length of MAC is fixed and depends upon the particular MAC algorithm.

When is a MAC secure? It should behave like a random function, for each key. Especially, it shouldn't be possible to predict new (m,t) pairs if you don't know k.

There are many examples of MACs. HMAC (a hash-based MAC) is one of the most common.

HMAC(m; k) = H(f1(k), H(f2(k), m))

Function H is a cryptographic hash function. It can be instantiated, for example, by any of the SHA-2 family. Functions f1 and f2 are specially designed to prevent certain attacks. Their exact details aren't important here—you can look them up if you're curious.

Another example is CBC-MAC, which uses CBC mode encryption to produce a tag.

Note that MACs do not protect confidentiality, at least not necessarily. Some happen to do so, but it's easy to construct MACs that don't.

Digital Signatures

A digital signature scheme is a set of algorithms for detecting modification of messages based on a public–private key pair. The public key for principal A, written K_A, is used to verify A's signatures. The private key for principal A, written k_A, is used by A to create signatures.

Digital Signature:

  1. (K_A, k_A) = Gen(len)
  2. A: s = Sign(m; k_A)
  3. A → B: m, s
  4. B: accept if Ver(m; s; K_A)

The digital signature scheme is the triple (Gen, Sign, Ver) of algorithms. Note that Ver takes three inputs: the message to verify, the purported signature on that message, and the verification key of the signer.

As with MACs, we want to be able to sign arbitrary length messages. But these Sign and Ver are public-key algorithms, which operate on big integers. So, as with public-key encryption, they are constrained to a limited input size.

In practice, messages are therefore hashed before being signed:

Digital Signature with Hashing:

  1. (K_A, k_A) = Gen(len)
  2. A: s = Sign(H(m); k_A)
  3. A → B: m, s
  4. B: accept if Ver(H(m); s; K_A)

Hashing is such a pervasive practice with signatures that, henceforth, we'll just assume the message is hashed without bothering to write that down as part of the protocol.

When is a digital signature scheme secure? It should work like hand-written signatures. In fact, it should be even better: an adversary shouldn't be able to forge signatures on new messages, even if given samples of other signed messages. Mathematically, the signature algorithm should behave, for each key, like a random function from messages to signatures.

Well-known examples of digital signature schemes include the following:

Digital Certificates

Here's one important use for digital signatures. Operating systems and browsers come preinstalled with digital certificates for companies such as Verisign. A digital certificate is means of associating a public key with a principal's identity. Let id_S be a string encoding the identity of a subject, let K_S be the subject's public (verification or encryption) key, and let k_I be the signing key of an issuer.

Digital Certificate:
I<<S>> = Sign(id_S, K_S; k_I)

I<<S>> is a digital certificate issued by I for S. It binds id_S to K_S according to I.

In practice, the most common format for certificates is X.509, an international standard. An X.509 certificate includes additional information, including a serial number for the certificate, a validity interval, etc.

We'll discuss digital certificates further when we cover authentication of machines.

Part 3: Confidentiality and Integrity

Authenticated Encryption

Suppose you want to protect both confidentiality and integrity. The result is called authenticated encryption. There are three generic ways of constructing authenticated encryption out of a standard block cipher and MAC. All three are used in real-world protocols.

There are also block cipher modes that are specifically designed to achieve both confidentiality and integrity. Galois/Counter Mode (GCM) is a popular choice, because it has high performance and isn't encumbered by patents.

Secure Sockets Layer (SSL)

Authenticated encryption is such a massively useful thing that it's long been available as part of libraries and other software distributions. Netscape introduced a protocol for it back in 1996 called Secure Sockets Layer (SSL) v3. SSL essentially provides authenticated encryption on top of TCP. SSL is used widely—for example, HTTPS is just HTTP run over SSL. SSL was standardized under the name Transport Layer Security (TLS), so you'll see it referred to by either name in the literature.

TLS manages sessions, which are bi-directional communication between a client and a server. The communication is optionally secured for both confidentiality and integrity against a Dolev–Yao attacker. Sessions are logical: there can be many sessions between any two physical hosts, and each host could be either client or server in any given session.

Each message sent during a session is called a record. Records are protected by MAC-then-Encrypt. The MAC used is HMAC. The hash function and encryption scheme used can be negotiated by the client and server for each SSL session. Digital signatures and certificates can used to negotiate the shared encryption and MAC keys for each session. We'll look more at the details of this negotiation when we discuss authentication of machines.

SSL/TLS in Java: Java provides JSSE (Java Secure Socket Extension). It's mostly very easy to use as a drop-in-replacement of standard network sockets. The hard part typically is management of digital certificates. Using JSSE in your course project is usually a Very Good Idea, assuming it satisfies your security goals. It's much better to reuse crypto code than to implement everything yourself.