Capturing kernel crash dumps with Juju

A little before the summer vacation, I decided that it was a good time to get acquainted with writing Juju charms.  Since I am heavily involved with the kernel crash dump tools, I thought that it would be a good start to allow Juju users to enable kernel crash dumps using a charm.  Since it means acting on existing units, a subordinate charms is the answer.

Theory of operation

Enabling kernel crash dumps on Ubuntu and Debian involves the following :

  • installing
    • kexec-tools
    • makedumpfile
    • kdump-tools
    • crash
  • Adding the crashkernel= boot parameter
  • Enabling kdump-tools in /etc/default/kdump-tools
  • Rebooting

On ubuntu, installing the linux-crashdump meta-package takes care of all the packages installation.

The crashdump subordinate charm does just that : installing the packages, enabling the crashdump= boot parameter as well as kdump-tools and reboot the unit.

Since this charm enable a kernel specific service, it can only be used in a context where the kernel itself is accessible.  This means that testing the charm using the local provider which relies on LXC will fail, since the charm needs to interact with the kernel.  One solution to that restriction is to use KVM with the local provider as outlined in the example.

Deploying the crashdump charm

the crashdump charm being a subordinate charm, it can only be used in a context where there are already existing services running. For this example, we will deploy a simple Ubuntu service :

$ juju bootstrap
$ juju deploy ubuntu --to=kvm:0
$ juju deploy crashdump
$ juju add-relation ubuntu crashdump

This will install the required packages, rebuild the grub configuration file to use the crashkernel= value and reboot the unit.

To confirm that the charm has been deployed adequately, you can run :

$ juju ssh ubuntu/0 "kdump-config show"
Warning: Permanently added '192.168.122.227' (ECDSA) to the list of known hosts.
USE_KDUMP:        1
KDUMP_SYSCTL:     kernel.panic_on_oops=1
KDUMP_COREDIR:    /var/crash
crashkernel addr: 0x17000000
current state:    ready to kdump
kexec command:
  /sbin/kexec -p --command-line="BOOT_IMAGE=/boot/vmlinuz-3.13.0-65-generic root=UUID=1ff353c2-3fed-48fb-acc0-6d18086d030b ro console=tty1 console=ttyS0 irqpoll maxcpus=1 nousb" --initrd=/boot/initrd.img-3.13.0-65-generic /boot/vmlinuz-3.13.0-65-generic

The next time a kernel panic occurs, you should find the kernel crash dump in /var/crash of the unit that failed.

Deploying from the GUI

As an alternate method for adding the crashdump subordinate charm to an existing service, you can use the Juju GUI.

In order to get the Juju GUI started, a few simple commands are needed, assuming that you do not have any environment bootstrapped :

$ juju bootstrap
$ juju deploy juju-gui
$ juju expose juju-gui
$ juju deploy ubuntu --to=kvm:0

 

Here are a few captures of the process, once the ubuntu service is started :

Juju environment with one service

Juju environment with one service

 

Locate the crashdump charm

Locate the crashdump charm

 

The charm is added to the environment

The charm is added to the environment

 

Add the relation between the charms

Add the relation between the charms

 

Crashdump is now enabled in your service

Crashdump is now enabled in your service

Do not hesitate to leave comments or question. I’ll do my best to reply in a timely manner.

Ce contenu a été publié dans Cloud, Juju, Ubuntu Server. Vous pouvez le mettre en favoris avec ce permalien.

Laisser un commentaire

Votre adresse e-mail ne sera pas publiée. Les champs obligatoires sont indiqués avec *