8 The Ensemble Security Architecture (by Ohad Rodeh)
This section describes the Ensemble security architecture. We
believe that Ensemble completely supports the fortress security
model. Only trusted, authorized members are allowed into the group.
Once a member is allowed into a group, it is completely trusted.
Ensemble is not secure against attacks from members that have been
admitted into the group: any group member can break the protocols by
sending bad messages.
The goal of our architecture is to secure group messages from
tampering and eavesdropping. To this end, all group messages are
signed and (possibly) encrypted. While it is possible to use public
key cryptography for this task, we find this approach unacceptably
expensive. Since all group members are mutually trusted, we share a
symmetric encryption key, and a MAC 1 key among them. These
keys are used to seal all group messages, making the seal/unseal
operation very fast2. As a shorthand,
we shall refer to the key-pair as the group key. Using a group key
raises two challenges:
-
A rekeying mechanism:
- allowing secure replacement of the current
group key once it is deemed insecure, or if there is danger that it
was leaked to the adversary. Dissemination of the new key should be
performed without relying on the old (compromised) group key.
- Secure key agreement in a group:
- i.e., a protocol that creates
secure agreement among group members on a mutual group key.
We focus on benign failures and assume that authenticated members will
not be corrupted. Byzantine fault tolerant systems suffer from
poor performance since they use costly protocols and make extensive use of
public key cryptography. We believe that our failure model is
sufficient for the needs of most practical applications.
The user may specify a security policy for an application. The policy
specifies for each address3
whether it is trusted or not. Each application maintains its
own policy, it is up to Ensemble to enforce it and
to allow only mutually trusted members into the same subgroup. A policy
allows an application to specify the members that it trusts and
exclude untrusted members from its subgroup.
8.1 Cryptographic Infrastructure
Our design supports the use of a variety of authentication and
encryption mechanisms. Ensemble has been interfaced with the OpenSSL
(see http://www.openssl.org/) cryptographic library, the PGP authentication engine, and the Kerberos
centralized authentication system (this is out of date). By default,
messages are signed using MD5, encrypted using RC4, and authentication
is performed using PGP. Because these three functionalities are
carried out independently any combination of supported authentication,
signature, and encryption systems can be used. A future goal is to
allow multiple systems to be supported concurrently. Under such a
system, processes would be able to compare the systems they have
support for and select any system that both have support for.
8.2 Rekeying
Ensemble rekeying uses the notion of secure channels. A secure
channel between endpoints p and q is essentially a symmetric
encryption key kpq agreed upon between p and q. This key is
known only to p and q and is different than the group
key. Whenever confidential information needs to be passed between p
and q it is encrypted using kpq and sent using Ensemble
reliable point-to-point messaging.
The basic rekeying protocol supported uses a binary tree structure.
In order to rekey the group, a complete binary tree spanning the group
is created. Member 0 is the father of 1 and 2, 1 is the father of 3
and 4, etc.. The leader chooses a new key knew and sends it securely to 1
and 2; member number 1 sends knew securely down to 3 and 4, etc.. When a tree
leaf receives a new key it sends up a clear-text acknowledgment. When
acknowledgments reach the leader (0) it prompts the group for a view
change in which the new key will be used.
knew is disseminated confidentially using secure channels. We
cannot use the old key to protect knew since the old key is
assumed to be compromised. Secure channels are created upon
demand by Ensemble, they are then cached for future use. Creating a secure
channel is a costly operation taking hundreds of milliseconds even on
fast CPUs. It is performed in the background so as not to block the
application.
Recently, we have added faster rekeying protocols to the system. A
complete implementation of the dWGL algorithm has been added, in the
form of several layers. There are two new algorithms rekey_dt, and
rekey_diam. There are described in the reference manual.
8.3 A secure stack
The Security architecture is comprised of 5 layers:
-
Exchange:
- secure key agreement. This layer is responsible for
securely handing the group key to new joining group components. Component
leaders mutually authenticate and check authorization policies prior
to handing the group key securely between them.
- Encrypt:
- chain-encryption of all user messages.
- Secchan:
- create and manage a cache of secure channels.
- PerfRekey:
- handling common rekeying tasks. For example, after a new
key has been disseminated to the group, acknowledgments must be
collected from all group members.
- Rekey_dt:
- Binary tree rekeying. Rekeying a group is very fast once secure
channels have been setup. We logged an average rekey operation for a 20 member
group at 100 milliseconds. Rekey_dt assumes that the Secchan and
PerfRekey layers are in the stack.
The regular and secure Ensemble stacks are depicted in
Figure ??. The Top and Bottom layer cap the stack
from both sides. The membership layers compute the current set of live
and connected machines, the Appl_top layer interfaces with the
application and provides reliable send and receive capabilities for
point-to-point and multicast messages. The RFifo layers provide
reliable per-source fifo messaging. The Exchange and Rekey layers are
related to the membership layers since the group key is a part of the
view information. The Encrypt layer encrypts all user messages hence
it is below the Appl_top layer.
Regular |
security additions |
Top |
|
|
Exchange |
|
Rekey_dt |
|
PerfRekey |
|
Secchan |
Gmp |
|
Top_appl |
Interface to the application |
|
Encrypt |
Rfifo |
|
Bottom |
|
Table 2: The Ensemble stack. On the left is the default stack that
includes an application interface, the membership algorithm and a
reliable-fifo module. To the right is a secure stack with the
Exchange, Encrypt, Rekey_dt, and Secchan layers in place.
8.4 Security events
There are three security events to note:
-
ERekey: By this event the application requests a Rekey operation.
-
ESecureMsg: This event is used by the Rekey layer to send private messages to
other processes. The Secchan layer catches this event
and sends the message securely to its destination.
-
ERekeyPrcl: this event is used in the communication between
all rekeying layers.
The Vs_key field was added to the view state was to allow for group keys.
It holds the current group key.
8.5 Using Security
Ensemble has three security properties:
-
Rekey: Add rekeying to the stack.
-
OptRekey: Use the dWGL algorithm for rekeying.
-
Auth: Authenticate all messages.
-
Privacy: Encrypt all user messages.
An application wishing for strong security should choose all of the
above properties in its stack and perform a Control Rekey action
once every several hours. Note that there are two flavors to
application Rekey-ing:
-
Rekey false: The default, as above.
- Rekey true: Cleanup prior to rekeying. For performance
considerations, Ensemble keeps cached key-ing material and secure
channels. These should be cleared up every couple of hours to prevent
an adversary from using cryptanalysis to discover the group key.
An example command line, for application appl, with pgp user name James_Joyce:
appl -add_prop Auth -add_prop Privacy -key 01234567012345670123456701234567
-pgp James_Joyce
In order to add authorization to the stack, thereby controlling which
members are allowed to join a group, one must do:
val policy_function : Addr.set -> bool
val interface : Appl_intf.New.t
let state = Layer.new_state interface in
let state = Layer.set_exchange (Some policy_function) state in
Appl.config_new_full state (ls,vs)
Instead of simply:
Appl.config_new interface state (ls,vs)
Authorization is not linked to the Security architecture, regular
stacks can perform authorization. Control of joining members is
delegated to the group leader that checks its authorization list and
allows/disallows join. Every view change the authorization list is checked and
existing members that are not authorized are removed.
In practice, if an application changes its authorization list
dynamically, it must perform a Prompt and a Rekey
whenever such a change occurs.
8.6 Checking that things work
To check that PGP has been installed correctly, that Ensemble can
talk to it without fault, and the cryptographic support is running
correctly, one can use the armadillo demo program.
In order to set up PGP, one must create principals and corresponding
public and private keys. These are installed by PGP in its local
key repository. The basic PGP key-generation command is:
zigzag ~/ensemble/demo> pgp -kg
To work with the armadillo demo, you'll need to create principals in
the group o1, o2, .... Armadillo creates a set of endpoints, and
then runs a test between them. To this end, the program has a ``-n''
flag that describes the number of endpoints to use. For example, the
command line armadillo -n 2 ... tells armadillo that use a two
members configuration. These members will have principal names o1
and o2 respectively.
To view the set of principals in the repository do:
zigzag ~/ensemble/demo> pgp -kv
pub 512/2F045569 1998/06/15 o2
pub 512/A2358EED 1998/06/15 o1
2 matching keys found.
To check that PGP runs correctly do:
zigzag ~/ensemble/demo> armadillo -prog pgp
PGP works
check_background
got a ticket
background PGP works
If something is broken, the PGP execution trace can be viewed using:
zigzag ~/ensemble/demo> armadillo -prog pgp -trace PGP
If more information is required use the flags -trace PGP1 -trace PGP2.
The default version of PGP that Ensemble works with is 2.6. If,
however, you'd like to use a different version, set your environment
variable ENS_PGP_VERSION to the version number. Versions 5.0 and
6.5 are also supported.
To check that OpenSSL is installed correctly, one can do:
zigzag ~/ensemble/demo> armadillo -prog perf
For a wider scale test use the exchange test. This is a test
that creates a set of endpoints, with principal names: o1, o2,
..., and merges them securely together into one group. Each group
merge requires that group-leaders properly authenticate themselves
using PGP. The test is started with all members in components containing
themselves, and ends when a single secure component is created.
Note that it will keep running until reaching the timeout. The timeout
is set by default to 20 seconds.
To invoke the test do:
zigzag ~/ensemble/demo> armadillo -prog exchange -n 2 -real_pgp
If something goes wrong, a trace of the authentication protocol is
available through -trace EXCHANGE.
The -real_pgp flag tells armadillo not to simulate PGP.
Simulation is the default mode for armadillo, since we use it to
test communication protocol correctness.
To check that rekeying works do:
zigzag ~/ensemble/demo> armadillo -prog rekey -n 5
To test security with two separate processes do the following:
zigzag ~/ensemble/demo> gossip &
zigzag ~/ensemble/demo> mtalk -key 11112222333344441111222233334444
-add_prop Auth -pgp o1
zigzag ~/ensemble/demo> mtalk -key 01234567012345670123456701234567
-add_prop Auth -pgp o2
The two mtalk processes should authenticate each other and merge.
The three command line arguments specify:
-
-key 111122223333444111122223333444 : The initial security key of the
system. Should be a 16 byte string.
- -add_prop Auth: Add the authentication protocol.
Otherwise, stacks with different keys will not be able to
merge.
- -pgp o1: Specify the principal name for the system.
8.7 Using security from HOT and EJava
The security options have been added to the HOT interface. For a
demonstration program look at hot_sec_test.c in the hot subdirectory.
The only steps one needs to make are: (1) Set the program's principal
name (2) Set the security bit. Both of these options are specified in the
join-options structure. For example, in hot_sec_test.c:
static void join(
int i,
char **argv
)
state *s ;
s = (state *) hot_mem_Alloc(memory, sizeof(*s)) ;
memset(s,0,sizeof(*s)) ;
s->status = BOGUS;
s->magic = HOT_TEST_MAGIC;
...
strcpy(s->jops.transports, "UDP");
strcpy(s->jops.group_name, "HOT_test");
...
sprintf(s->jops.princ, "Pgp(o%d)",i);
s->jops.secure = 1;
...
/* Join the group.
*/
err = hot_ens_Join(&s->jops, &s->gctx);
if (err != HOT_OK)
hot_sys_Panic(hot_err_ErrString(err));
EJava is interfaced with HOT, so they share a similar interface. Note
that the outboard mode, supported by both interface is insecure. The messages passing on the TCP connection between the
client and server are neither MACed nor encrypted. Therefore, they can
be used securely only when situated on a single machine.