Quickstart Guide (2011-1)

Welcome to Nexus, a research OS with unique mechanisms for attestation and authorization. Nexus is also a microkernel operating system with a Unix environment that supports well known tools such as MPlayer, Python and Busybox. It should feel fairly familiar to seasoned Unix users -- even if as a research OS it can be a bit rough around the edges.

Table of Contents

Quickstart
Booting into the Shell
Applications
Building from Source
Further information

About this guide

This guide applies to version 2011-1. Help with older releases can be found here.

Throughout the text, asides are encapsulated
in a gray box
commands are rendered as
monospace text in a red box

Quickstart

The easiest way to get started with Nexus is to download the VMWare virtual machine bundle, which consists of configuration file nexus-2011-1.vmx and virtual disk image nexus-2011-1.vmdk. Download the free VMWare player and open the vmx file.

Qemu and others

To run Nexus on real hardware, in Qemu or in VirtualBox, get the livecd nexus-2011-1.iso, instead. The simplest qemu command is
qemu -m 512 -cdrom nexus-2011-1.iso /dev/null
This version differs slightly from the demo discussed here. Most importantly, the livecd lacks directory /usr. For more information on booting see the files doc/HOWTO.boot and doc/HOWTO.qemu in the sourcecode.

Booting into the Shell

Nexus boots up in a few seconds, most of which is spent running self tests. After kernel boot, the system starts the Nexus shell. This proceeds to load a set of applications, including device drivers and a webserver. While the applications are loading in the background, have a look at the basic Nexus interface.

Console

The screen is split vertically into two regions by a blue bar. The large bottom region presents the currently active console, similar to most operating systems. The top region is a protected section that only the operating system can write to. Here, the kernel writes the number of the current console and the name of the foreground application on the right-hand side. For non-system tasks, the left-hand side displays the SHA1 hash of the application binary. Nexus only runs statically linked applications, ensuring that the hash covers all code.

By default, each application opens into a private virtual console. [Alt]-[Tab] navigates between consoles, and [Alt]-[Tab]-[Shift] does the same in inverse direction. Console 0 is always a read-only environment that displays the kernel log. Console 1 shows the operations and decisions of the system guard process. The shell occupies console 2.

Login and Access Control

Unlike multi-user operating systems, Nexus does not require you to login with a username and password. This is a good time to mention that although Nexus is Unix-like, it strays from common practice where this clashes with research. Most importantly, it forgoes Unix-style access control based on permissions for user, group and other. A discussion of authorization in Nexus is beyond the scope of this document; assume that Nexus performs no access control checks unless an application explicitly sets a guard. Therefore, the shell performs no user authentication at present.

Shell Commands

Once the shell has finished booting background applications, it writes [sh] ready. The Nexus shell is a rudimentary environment for browsing the filesystem and starting applications. It offers a small list of built-in operations. Use
help
for more information. It also offers a few kernel operations, such as
ps
The command
kernel help
lists these. Finally, you can start applications, which will open and jump to a new console. To execute an application in the background of the current console, append the command line with an ampersand (&), for instance
usr/bin/lighttpd -D -f usr/etc/lighttpd.conf &

Help out:Bourne Shell

Besides the native shell, Nexus also ships with a more complete Bourne shell clone: dash is compiled into the busybox application. Before dash can be used, however, a few Unix interfaces must be implemented. This is a great project to get started hacking on Nexus. In particular, dash expects a rudimentary Posix signals implementation.

Shortcuts

The function keys offer shortcuts to some kernel commands. [F1] dumps per-process memory usage information. [F3] starts a background process cycle counter, which prints usage summaries every 8 seconds to the foreground console. [F4] toggles logging of the kernel guard (a critical part of Nexus authorization). The other keys are still unassigned. Because it is a research OS, Nexus offers a very easy way to reboot: the [ESC] key causes an immediate power cycle.

Contents & Directory Structure

The shell lands you at the root of the filesystem, which has the usual entries:

/bin applications
/dev
/etc
/mnt
/proc live metadata 'introspection' filesystem
/tmp
/usr larger filesystem: fat32 or nfs
/var logfiles, among other non-volatile data

The root filesystem is generated by the kernel from an initial ramdisk, but as a microkernel, Nexus pushes most storage to external fileserver applications. The kernel server only holds data in main memory. /bin contains the files that were shipped on the ramdisk. The vmware image then mounts a non-volatile filesystem at /usr from the .vmdk disk image. For this purpose, the shell loads the IDE disk driver /bin/stor-ide.drv and the Fat32 fileserver /bin/fat32.app. On the .iso livecd, these commands are absent. In general, such boot commands are recorded in the file initscript in the initial ramdisk.

Besides the Fat32 driver, Nexus also ships with a Network Filesystem (NFS) server. For development, it is often faster to mount the filesystem from the development host over the network than regenerating hard drive images on each change.

The vmware demo image holds a complete copy of the sourcecode in /usr/src

Applications

The demo distribution ships with the most popular Nexus applications. To enable others, read the build instructions below and compile a custom version.

Device Drivers

In a microkernel, device drivers are moved out of the kernel into their own isolated processes. The vmware demo image is configured to automatically load all drivers that match the vmware hypervisor.

The keyboard and ps/2 mouse drivers are bundled in bin/kbd.drv. Similarly, Nexus bundles network drivers for popular Intel, Broadcom and AMD devices, corresponding to the Linux e1000, tg3 and pcnet32 drivers, in bin/net.drv. It also supports alternative in-kernel versions of these. The VESA video driver is kernel only (for now: moving this to userspace is another nice engineering task). We have already seen the IDE storage driver bin/stor-ide.drv.

As basis for trustworthy attestation, Nexus supports devices that can act as hardware root of trust: a relatively tamperproof source of code measurement. For this purpose, it incorporates drivers for the Trusted Platform Module (TPM) devices common in the pc platform. bin/tpm.drv bundles drivers for Atmel and National Semiconductor TPM 1.1 devices. For environments that lack a TPM, such as the VMWare virtual machine, bin/tpmd emulates a TPM 1.2 device (with thanks to the Bochs developers).

Unix tools

The core Unix environment is provided by Busybox. The version in
/usr/bin/busybox
is compiled with many tools, including
/usr/bin/busybox ls
and
/usr/bin/busybox cat /usr/src/README
Many have not, or not extensively, been tested, and may prove to expect Unix library functions or system calls that Nexus does not yet provide. If this happens, the system will print a message "NXLIBC ERROR: syscall #[0-9]+ not supported. Aborting" and abort the calling process.

Python

We have ported Python 2.6.4 together with sqlite and berkeleydb, among others. Start the application using
/usr/bin/python -i
to enter interactive mode. It takes Python a few seconds to load all modules from disk.

Webserver

The demo automatically starts a webserver that gives access to the sourcecode. At startup, look for the message from the dhcp client about obtained IP address, then from another host navigate to port 80 at this address or to port 443 for secure HTTPS connections. The demo runs Lighttpd to serve pages and communicate with Python. Nexus also ships with a homegrown server
/usr/bin/httpd.app
that has some Nexus-specific features.

Other

The subdirectory /usr/bin/tpm/ holds the common TPM tools from the libtpm environment.
/usr/bin/tpm/tpm_demo
runs a simple testsuite. More applications can be included by building code from source.

Building from source

The demo lacks some applications and is compiled with full (debugging) information. To customize Nexus, download the sources, either from git at git://git.systems.cs.cornell.edu/nexus or the source tarball that corresponds to the demo. Read the /INSTALL file in the root of the distribution for information about prerequisite packages on the development host. Then, run make menuconfig to configure the system, followed by make; make.

Configuration

Nexus uses the menu-driven configuration system make menuconf known from the Linux kernel. Most options are fairly self explanatory. The menusystem allows selection of third party packages for inclusion. Some packages (such as Python) only appear when all their prerequisites have been selected.

Besides the menu-driven build options, Nexus stores compile options in the form of CPP preprocessor macros in common/include/nexus/config.h.

Source Directory Layout

Nexus sourcecode is split between userspace code in /user, kernel code in /kernel and code that compiles to both environments in /common. Within these, further divisions separate applications (/user/apps), drivers (/user/drivers and /kernel/drivers) and more. /INSTALL gives more details.

Installation

During compilation, a complete copy of the source tree is made under /build. After compilation, the build directory build/boot/stage1 holds a readymade live cd image nexus.iso or a vmware or qemu image, depending on the target selected during make menuconfig. To install Nexus from these images, consult the files /doc/HOWTO.boot, /doc/HOWTO.vmware and /doc/HOWTO.qemu. Nexus can also be booted over the network using GPXE boot images. The file doc/HOWTO.gpxe explains this process.

Demo Idiosyncracies

The demo differs a little bit from a standard Nexus environment built from source. It (1) always executes all self tests, which adds a few seconds of boot time, (2) comes with a disk image that contains the entire source tree for reference and (3) runs the system guard process in debug mode, so that it logs all operations and decisions to a console.

Further Information

The /doc directory in the sourcecode holds additional tips on specific technical topics, such as how to use NFS or enable debugging with gdb. The design goals and abstract system architecture are presented in talks and papers. A paper about the design of the OS proper is still in preparation. Contact us directly for details, and also for all your bug reports, questions and remarks.

Known Issues

The file BUGS in the source code lists the critical issues. Besides, these, Nexus lacks some features that you may take for granted:

PWD Nexus does not pass environment variables to child processes on creation. The main issue you may notice is that as a result of this each application has its initial working directory (pwd) set to the filesystem root, not to the directory of hits parent process. To illustrate: calling /usr//bin/busybox cat README from /usr/src will fail, because busybox will search for /README. Instead, use absolute paths, as in /usr/bin/busybox cat /usr/src/README.
Environment passing is actually very easy to fix. If you want to help out, this is a great place to start. Contact us for details.
DNS Name resolution fails in most cases. Always pass IP addresses to applications instead of hostnames, even if the application claims to support hostname resolution. Most applications rely on the lookup interfaces of uClibc, which requires some OS support that Nexus lacks.