LXC with Gentoo Linux
VMWare (Server, Player, ESX), VirtualBox, KVM/QEMU are just a few of the virtualization technologies you can use if you need to run a generic virtual machine. Each has strong and weak points, each adds some overhead when running the virtual machine, each can prove useful in a given scenario. I’ve used more or less extensively all of them (testing or in daily work). I’ve tested also linux specific virtualizations like XEN or OpenVZ but their drawbacks made me somewhat reluctant in using them. But recently I’ve come over Linux Containers (LXC), a lightweight virtualization that lets you isolate processes and resources without using a full virtual machine.
Please note that LXC does not provide you a full virtual machine and does not let you run another OS than Linux. It is similar with XEN or OpenVZ but offers greater flexibility being integrated in the kernel (don’t require additional patches or changes to the system). If you wish to know more about how it works and why it’s better than a full virtual machine please see here, I’ll continue with a short how-to for configuring and using it in Gentoo Linux.
First of all you’ll need a recent kernel as LXC is integrated starting from version 2.6.29. I’m using 2.6.34 and the configuration options are relative to it. Please make sure the following configuration parameters are set:
If you have made changes to your kernel configuration you’ll need to rebuild, install and use the new kernel. For Gentoo you can use the information in the kernel upgrade documentation about how to do it right.
Next you will have to mount the control group filesystem. For this execute:
# mkdir -p /cgroup
# mount none -t cgroup /cgroup
To persist the change between reboots add to /etc/fstab:
none /cgroup cgroup defaults 0 0
Install the LXC userspace tools:
# emerge -v lxc
Configure a bridge to be used by the container. In my case I created a separate virtual network which is given access to the real network on a as-needed bases. This way I can communicate with the containers (and the containers between them) but I can control what container has access to the real network and when. In /etc/conf.d/net add:
config_br0="10.1.1.1/32 brd 10.1.1.255"
routes_br0="10.1.1.0/24 via 10.1.1.1"
and create the /etc/init.d/net.br0 symlink to /etc/init.d/net.lo. To allow a container to communicate with the world just activate the NAT into iptables like
# iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
where eth0 is the real network interface.
Start installing the container by creating the root folder
# mkdir -p /mnt/storage/containers/gentoo_04/roofs
The rest of the steps are the same as for a new Gentoo installation. Please refer to the Gentoo Handbook for them, I’ll give here only the most important ones and what is made different from the standard. First download a stage3 file and a portage snapshot (chapter 5.b, 5.c from the handbook) into rootfs
# cd /mnt/storage/conainers/gentoo_04/rootfs
Donwload from a mirror and unpack the 2 files
# tar xvjpf stage3-*.tar.bz2
# tar xvjf portage-latest.tar.bz2 -C /mnt/storage/containers/gentoo_04/rootfs/usr
Configure compile flags, select mirrors and enter into the new environment
# mirrorselect -i -o >> /mnt/storage/containers/gentoo_04/rootfs/etc/make.conf
# mirrorselect -i -r -o >> /mnt/storage/containers/gentoo_04/rootfs/etc/make.conf
# cp -L /etc/resolv.conf /mnt/storage/containers/gentoo_04/rootfs/etc/
# mount -t proc none /mnt/storage/containers/gentoo_04/rootfs/proc
# mount -o bind /dev /mnt/storage/containers/gentoo_04/rootfs/dev
# chroot /mnt/storage/containers/gentoo_04/rootfs /bin/bash
# source /etc/profile
# export PS1="(chroot) $PS1"
# emerge --sync
Now the container is nearly ready (we don’t need to configure and install the kernel as we’ll use the host one. What we need is to switch to OpenRC by following the migration guide.
# mkdir /etc/portage
# echo "sys-apps/openrc ~x86" >> /etc/portage/package.keywords
# echo "sys-apps/baselayout ~x86" >> /etc/portage/package.keywords
# emerge -v openrc mc
Don’t forget to change the root password otherways you won’t be able to login int your container and will need to reenter chroot and change it (or hack the shadow file :) )!
You can exit the chroot at this step.
Create the container definition file in /mnt/storage/containers/gentoo_04/lxc-gentoo.conf
lxc.utsname = gentoo_04
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.name = eth0
lxc.mount = /mnt/storage/containers/gentoo_04/fstab
lxc.rootfs = /mnt/storage/containers/gentoo_04/rootfs
Create the fstab file /mnt/storage/containers/gentoo_04/fstab
none /mnt/storage/containers/gentoo_04/rootfs/dev/pts devpts defaults 0 0
none /mnt/storage/containers/gentoo_04/rootfs/proc proc defaults 0 0
none /mnt/storage/containers/gentoo_04/rootfs/sys sysfs defaults 0 0
none /mnt/storage/containers/gentoo_04/rootfs/dev/shm tmpfs defaults 0 0
No changes are required for the container devices in /dev.
Update the /etc/inittab file by commenting the lines from the TERMINALS section c1 to c6
#c1:12345:respawn:/sbin/agetty 38400 tty1 linux
and adding the following line
1:2345:respawn:/sbin/agetty 38400 console
Next update the container network configuration in /mnt/storage/containers/gentoo_04/rootfs/etc/conf.d/net
config_eth0="10.1.1.4 netmask 255.255.255.0 brd 10.1.1.255"
routes_eth0="default via 10.1.1.1"
Update the runlevels to remove unneeded/unsupported services
- remove consolefont, hwclock, keymaps from /etc/runlevels/boot
- remove udev-postmount from /etc/runlevels/default
- remove udev from /etc/runlevels/sysinit
Now you are ready to create the container
# lxc-create -f /mnt/storage/containers/gentoo_04/lxc-gentoo.conf -n gentoo_04
If you need to make a change to the configuration files the fastest way is to destroy the container, make the changes and recreate it
# lxc-destroy -n gentoo_04
Running the container
In order to start the container you have to use
# lxc-start -n gentoo_04
At this step, if everything is succesfull you will be greeted with a login prompt. Enter root and your password (did you forget to set it? then enter a chroot environment and reset it) and you are in. Now you should enable networking and add it at default runlevel
# /etc/init.d/net.eth0 start
# rc-update add net.eth0 default
You should finish configuring the new system as per chapter 9 in handbook (install system tools, add user).
In order to shutdown you can do it in the usual way: issue a poweroff at the prompt. The system will shutdown but will not exit. To finish for good you’ll need
# lxc-stop -n gentoo_04
in another prompt (and you should run reset in the console used by the container). You can issue the stop command even if the container is still running but in this case you’ll lose any usaved information.
Now, that you have the container ready you can proceed to install any software you need. I’m using it for running apache, php, mysql, postgresql, oracle database and various other servers used in a development environment. When I don’t need a particular container I just destroy it and delete the rootfs folder. In this article I’ve show how to install a Gentoo container on a Gentoo host but you can run other linuxes in the container. You can find the instructions for other platforms in the references section below.