[Lab 0 Part B] Working with Xen-Blanket
If you have not finished [Lab 0 Part A], please finish it first. This part of the lab requires the knowledge and materials from the first part. You should use your own security group, keypairs, and so on from Part A, which are indicated as gp-netid-xxx, kp-netid-xxx, or id-rsa-kp-netid-xxx in this webpage.
In this lab you will install and run Xen-Blanket in Amazon EC2 and bundle the image. Xen-Blanket is a virtual machine monitor (VMM) that runs on a VMM. Thus, running a Xen-Blanket on Amazon EC2 is nesting two virtualized layers in the cloud. For more information about Xen-Blanket, visit http://xcloud.cs.cornell.edu.
1. Building and Installing Xen-Blanket in EC2
These instructions are for running the Xen Blanket on an Amazon EC2
cluster instance. The key components are a patched Xen and blanket
drivers implemented as Linux modules.
Before you start an Amazon EC2 instance, please mark that you are using the instance in the shared document. The link to the document can be found in CMS. We have limited shared budget to access EC2, so we don't want any instance to be running when you are not actually doing something on it. Instances running without the usage indication in the shared document can be terminated without prior notice.
Launch Amazon EC2 instance EC2 CentOS 5.4 HVM AMI (ami-7ea24a17). The instance must be set to cluster compute instance (using option -t cc1.4xlarge).
aws run-instances ami-7ea24a17 -g gp-netid-xxx -k kp-netid-xxx -t cc1.4xlarge
Log into the Amazon instance using a ssh client. You may have to wait for a while for the instance to be completely booted.
ssh -i ~/.aws/id-rsa-kp-netid-xxx firstname.lastname@example.org
Download and unpack the Xen-Blanket patches (xen.blanket.1.0.1.tar.gz), then enter the release directory.
tar zfx xen.blanket.1.0.1.tar.gz
We must next obtain the third party Xen and Linux sources that we
will apply the Xen-Blanket patches against. They are from the
CentOS distribution; the original source RPM
(kernel-2.6.18-194.26.1.el5.src.rpm) can be found in the CentOS
vault at http://vault.centos.org/5.5/updates/SRPMS/. For
convenience, we have already assembled the source RPM into a
tarball, available at http://xcloud.cs.cornell.edu. From the
release directory, download the source tarball, then run the
create_source_tree script. The create_source_tree script patches
the supplied kernel and Xen, and also copies and patches PV-on-HVM
drivers and virtio drivers to make them work as blanket drivers.
We have now patched the Linux and Xen source to include support for
blanket drivers. Next, we must install the Xen tools, build and
install the blanket-enabled Xen, and build and install blanket
driver modules. The modules can be built against a ready-built
Linux kernel or we can build a kernel from scratch. Yum can be
used to avoid building most things from scratch. To be sure that
packages obtained from yum are compatible with the patched code
version, we first change the yum repositories to point to an older
repository by copying the included .repo file. The following yum
line installs the Xen tools, a xen-enabled Linux kernel for domain
0, and Xen. In the next step, we will be replacing the installed
Xen with one that supports blanket drivers.
If you are using xen-blanket version 1.0.0, there is a minor error to fix in the CentOS-Base-Blanket.repo file.
You should open the file with a text editor and remove the first
line that says,
[root@ip-10-17-185-112 kernel-2.6.18]# cat /etc/yum.repos.d/CentOS-Base.repo.
Now you are ready to proceed. Execute the following commands.
cp CentOS-Base-Blanket.repo /etc/yum.repos.d/CentOS-Base.repo
yum clean all
yum install -y xen xen-devel xen-libs xen-libs-devel
kernel-xen kernel-xen-devel gcc
Make sure the PV drivers will be loaded for the network and the
disk on boot by putting aliases in the /etc/modprobe.conf file.
The network alias should already be there.
#alias scsi_hostadapter ata_piix
alias scsi_hostadapter xen-vbd
alias eth0 xen-vnif
We must still build and install the blanket-enabled Xen and the
blanket driver modules for Linux.
First we enter the patched source directory.
By default, the Makefile is set up to build a Linux kernel from
scratch. To avoid this, edit the Makefile to build the blanket
drivers against the second-layer domain 0 Linux kernel installed two steps before.
For example, to find out the name of the kernel
sources, and the kernel version for modules run:
# ls /usr/src/kernels/|grep el5xen
# ls /lib/modules/ |grep xen
and then change the KERNEL_SRC and KERNEL_VERSION lines in the
Makefile to match:
Run make to build and install
Xen and the blanket drivers for Xen (Amazon is Xen based):
Replace the /boot/grub/menu.lst with the code below. It sets the default to the Xen
entry, and replace the xen line with the newly installed Xen. It replaces the linux
images to load when system reboots. Make sure the kernel and module names names match the ones
in your /boot folder.
serial --unit --speed=57600
terminal --timeout=1 console serial
title CentOS (2.6.18-238.9.1.el5xen)
kernel /xen-3.1.2-blanket.gz com1=115200,8n1 console=com1
module /vmlinuz-2.6.18-194.32.1.el5xen ro root=/dev/VolGroup00/LogVol00 rhgb quiet console=hvc0 earlyprintk=xen ide0=noprobe
Change the default network setup
behavior of Xen away from bridging, in
/etc/xen/xend-config.sxp. NAT is one option.
Comment out the following lines using #.
Add the following new lines.
You are almost done. Before you reboot check your linux version.
Linux version 2.6.18-164.15.1.el5 (email@example.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-46)) #1 SMP Wed Mar 17 11:30:06 EDT 2010
Copy the information or remember the linux version so that you can compare it with the version that you will see once the xen-blanket starts to run.
Now Reboot! When the instance comes back up, you should be able to
log into the second-layer domain 0. Be patient. Rebooting takes quite a while. If you can
connect and log in, the blanket drivers are working. Check the linux version using the same command from the previous step. You will see a different version number.
Linux version 2.6.18-194.32.1.el5xen (firstname.lastname@example.org) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-48)) #1 SMP Wed Jan 5 18:44:24 EST 2011
2. Bundling Xen-Blanket Image as EBS Image
Once you have verified that you are done with the previous steps, you should bundle your image. Different from Lab 0 Part A, you should
use different commands to bundle the image, because the image size of cc1.4xlarge instance exceeds the standard size (10GB). The image
must be saved as an elastic block storage (EBS) image, which supports up to 1TB.
To save your instance as an EBS image, you should first figure out your instance id
using command while your instance is running (Note that "prompt>" is for the commands typed in your local machine, not in the instance in Amazon EC2. Replace netid with your own netid..)
prompt> aws describe-instances | grep netid
The instance id is the first returned value in the format of i-xxxxxx. Now you can bundle your image using the following command with the instance id. You should leave the instance to be running at this point.
prompt> ec2-create-image -n "im-netid-xen-blanket" i-xxxxxx
This command will immediately return an image id (ami-xxxxxxx), but you won't be able to connect to your instance for a while.
Wait for several minutes then you will be able to connect back to instance using ssh. If you want to check whether your image is ready
you can use the command,
prompt> ec2-describe-images -o self |grep ami-xxxxxxx
This will return whether your image is available or pending. When the state becomes available, your image is ready for use.
For this image you don't have to modify the manifest file as you did in Lab 0 Part A.
3. Terminate Instance
As the final step DO NOT FORGET TO TERMINATE THE INSTANCE.
Please understand that the large EC2 instance that you are using for this part costs much more than the one used in part A.
aws terminate-instances i-xxxxxx
Also unmark the usage indication in the shared document.
4. End of Lab 0
At this point you are done with Lab 0! Double check that all your instances are terminated, and two ami images from both Part A and B are properly created in EC2.
Most of the instructions in this page came from http://xcloud.cs.cornell.edu/code/README.txt