Roxy Pony's blog

Only fresh pony stuff

Aufs

September 17, 2015 / by Guillaume Petit

AUFS storage driver with Docker on OVH

Most linux distributions come with the proper support for AUFS as a Storage driver for docker. But OVH uses a custom kernel that just doesn’t. But no worries, today we will see how to change that !

Switching kernel

If you ever wonder how to know which kernel you are running on type $ uname -r. At the time of this tutorial my kernel was 3.14.32-xxxx-grs-ipv6-64. So the first thing we need to do, is to grab the list of the available images.

$ apt-cache search linux-image-extra

linux-image-extra-3.19.0-15-generic - Linux kernel extra modules for version 3.19.0 on 64 bit x86 SMP  
linux-image-extra-virtual - Transitional package.  
linux-image-extra-3.19.0-16-generic - Linux kernel extra modules for version 3.19.0 on 64 bit x86 SMP  
linux-image-extra-3.19.0-18-generic - Linux kernel extra modules for version 3.19.0 on 64 bit x86 SMP  
linux-image-extra-3.19.0-20-generic - Linux kernel extra modules for version 3.19.0 on 64 bit x86 SMP
linux-image-extra-3.19.0-21-generic - Linux kernel extra modules for version 3.19.0 on 64 bit x86 SMP    
linux-image-extra-3.19.0-22-generic - Linux kernel extra modules for version 3.19.0 on 64 bit x86 SMP  
linux-image-extra-3.19.0-23-generic - Linux kernel extra modules for version 3.19.0 on 64 bit x86 SMP
linux-image-extra-3.19.0-25-generic - Linux kernel extra modules for version 3.19.0 on 64 bit x86 SMP
linux-image-extra-3.19.0-26-generic - Linux kernel extra modules for version 3.19.0 on 64 bit x86 SMP

Select the most advanced version in the list, in my case here linux-image-extra-3.19.0-26-generic, so let’s install it !

$ apt-get install linux-image-extra-3.19.0-26-generic

Now we upgrade grub, to make this image appears in its configuration

$ update-grub

But we did not change anything yet, as the boot order will set our new image after the one from OVH. Let’s dive into the grub conf to change that.

$ fgrep menuentry /boot/grub/grub.cfg

if [ x"${feature_menuentry_id}" = xy ]; then  
  menuentry_id_option="--id"  
  menuentry_id_option=""  
export menuentry_id_option  
menuentry "Ubuntu 14.04 LTS, OVH kernel 3.14.32-xxxx-grs-ipv6-64" {  
menuentry 'Ubuntu' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-simple-c990f64a-61ea-4a90-b15c-c116cb188e99' {  
submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-c990f64a-61ea-4a90-b15c-c116cb188e99' {  
  menuentry 'Ubuntu, with Linux 3.19.0-26-generic' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-3.19.0-26-generic-advanced-c990f64a-61ea-4a90-b15c-c116cb188e99' {  
  menuentry 'Ubuntu, with Linux 3.19.0-26-generic (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-3.19.0-26-generic-recovery-c990f64a-61ea-4a90-b15c-c116cb188e99' {  
menuentry 'Memory test (memtest86+)' {  
menuentry 'Memory test (memtest86+, serial console 115200)' {

Perfect, we see both images, the one from OVH itself Ubuntu 14.04 LTS, OVH kernel 3.14.32-xxxx-grs-ipv6-64 but more importantly, the one we just installed. It’s a bit tricky to see as it’s in a submenu. You see the line with Ubuntu, with Linux 3.19.0-26-generic under the submenu Advanced options for Ubuntu ? Great ! That’s was we are looking for !

Grub will boot our kernel from the first image in that list, unless asked to do otherwise. So we have to update its settings to make that swap.

$ nano /etc/default/grub

You want to update the line GRUB_DEFAULT='' and set the image. The syntax value of this ENV variable is a bit special, it has to be like that SUBMENU_NAME>MENUENTRY_NAME. In our case, the submenu is Advanced options for Ubuntu and the menuentry is Ubuntu, with Linux 3.19.0-26-generic. So the line in the configuration file will end up like that

GRUB_DEFAULT="Advanced options for Ubuntu>Ubuntu, with Linux 3.19.0-26-generic"

Save and now it’s time to reboot your server

$ reboot now

Once your server is back, log into it and type $ uname -r to make sure the change was effective. It should display the version of our new image.

We are now ready to install docker !

Install docker

$ curl -sSL https://get.docker.com/ | sh

Once installed, make sure everything works fine with

$ docker run hello-world

And you can check that now docker is properly using AUFS

$ docker info

Containers: 27  
Images: 633  
Storage Driver: aufs  
  Root Dir: /home/docker/aufs  
  Backing Filesystem: extfs  
  Dirs: 687  
  Dirperm1 Supported: true  
Execution Driver: native-0.2  
Logging Driver: json-file  
Kernel Version: 3.19.0-26-generic

Perfect !

Bonus stage !

The rest of this tutorial will be more focused on specific tweaks regarding docker. You won’t need them to use docker, but they could interest you anyway !

Changing docker working directory

By default docker will put everything (images and containers) in /var/lib/docker. The problem with docker is disk usage, as it tends to eat a lot of space. On our server, we have two disks, a small one (20Go) and a very big one (2To), and /var/lib/docker is in the small one, so we had to move docker somewhere else.

To find a better suitable place, we had to know where our disks are mounted

$ df -u

Filesystem      Size  Used Avail Use% Mounted on
udev            7.9G     0  7.9G   0% /dev
tmpfs           1.6G  167M  1.5G  11% /run
/dev/sda2        20G  2.1G   17G  12% /
tmpfs           7.9G  3.2M  7.9G   1% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           7.9G     0  7.9G   0% /sys/fs/cgroup
/dev/sda3       1.8T   37G  1.7T   3% /home
tmpfs           1.6G     0  1.6G   0% /run/user/1000
tmpfs           1.6G     0  1.6G   0% /run/user/0

As you can see, the small disk (sda2) is mounted at / while the big one (sda3) at /home. So we had to move docker somewhere under /home. To make it simple we moved it into /home/docker.

To do so, we will have to modify the docker settings. By default docker put a sample configuration file in /etc/default/docker, yet this configuration is not used anyhow when the service is run. We will first update this file, then make sure docker uses it when it starts!

$ nano /etc/default/docker

You will see a couple of examples, but all we need here is to add the following line

DOCKER_OPTS="-g /home/docker/"

Yes, the -g option is to set the working directory !

We now need to update how systemctl starts docker so that it uses this configuration file.

$ nano /lib/systemd/system/docker.service

Most likely you will have to add the EnvironmentFile in the [Service] tag and to modify the ExecStart to add the $DOCKER_OPTS we set in the previous configuration file.

In the end your file will look like that

...
[Service]
EnvironmentFile=-/etc/default/docker
ExecStart=/usr/bin/docker -d $DOCKER_OPTS -H fd://
...

If you wonder, the dash - in front of /etc/default/docker is not a typo, you need it ! Now if we ever update the configuration file in /etc/default/docker and restart the service, docker will automatically use those new settings !

Now let’s update systemctl and restart our service

$ systemctl daemon-reload

By the way, you cannot move and use your former images/containers from /var/lib/docker. You will have to start from scratch ! Don’t try to move the former folder around as containers are configured with hardcoded paths pointing to /var/lib/docker.

$ systemctl stop docker

We can now clean up the former folder and setup the new one.

$ rm -rf /var/lib/docker
$ mkdir /home/docker

We can restart docker

$ systemctl start docker

And make sure to enable docker when your server starts

$ systemctl enable docker

With that you should be all set, enjoy !

References

This guide was built using various resources from here and there

Aufs setup

Docker install

Docker configuration

Image