While Heat exists mostly as an orchestration tool for OpenStack, it is also an interesting system for describing in templates interactions with APIs. There has been a resource to talk to the Docker API in Heat for a few months now [1], and we’ve seen some great examples of how to to use it. Most of them expect a Docker deployed on your Heat node, with all your users talking to it. With collegues, we thought about how you could use Nova servers with Docker installed and talk to the remote API [2]. This way we get a tenant-specific Docker instance on which we have full control.
While exploring those capabilities, I discovered that Docker introduced several exciting features that would make the template description much nicer. I wrote a patch to be able to use them [3], the following example relies on it, the good news being that it got recently merged in Heat master branch. You also need to enable the Docker resource in your heat deployment [4].
The main issue to solve when trying to deploy such a template is to make sure that your Docker service is ready before starting to create the containers in it. The new supported way to do this is to use software deployments resources. You need tools on your base image to talk to Heat, as used by TripleO. Building such an image is described in the heat-templates repository [5].
We’ll build the following example, creating our Docker server and 2 containers inside for having a WordPress deployment:
This is what an example template looks like:
heat_template_version: 2013-05-23 description: > Heat Docker template using software deployments. parameters: key_name: type: string description : Name of a KeyPair to enable SSH access to the instance default: heat instance_type: type: string description: Instance type for WordPress server default: m1.small image: type: string description: > Name or ID of the image to use for the Docker server. This needs to be built with os-collect-config tools from a fedora base image. resources: docker_sg: type: OS::Neutron::SecurityGroup properties: description: Ping, SSH, Docker rules: - protocol: icmp - protocol: tcp port_range_min: 22 port_range_max: 22 - protocol: tcp port_range_min: 80 port_range_max: 80 - protocol: tcp port_range_min: 2345 port_range_max: 2345 docker_config: type: OS::Heat::SoftwareConfig properties: group: script config: | #!/bin/bash -v setenforce 0 yum -y install docker-io cp /usr/lib/systemd/system/docker.service /etc/systemd/system/ sed -i -e '/ExecStart/ { s,fd://,tcp://0.0.0.0:2345, }' /etc/systemd/system/docker.service systemctl start docker.service docker_deployment: type: OS::Heat::SoftwareDeployment properties: config: {get_resource: docker_config} server: {get_resource: docker_host} docker_host: type: OS::Nova::Server properties: image: {get_param: image} flavor: {get_param: instance_type} key_name: {get_param: key_name} security_groups: - {get_resource: docker_sg} user_data_format: SOFTWARE_CONFIG database_password: type: OS::Heat::RandomString database: type: DockerInc::Docker::Container depends_on: [docker_deployment] properties: image: mysql name: db docker_endpoint: str_replace: template: http://host:2345/ params: host: {get_attr: [docker_host, networks, private, 0]} env: - {str_replace: {template: MYSQL_ROOT_PASSWORD=password, params: {password: {get_attr: [database_password, value]}}}} wordpress: type: DockerInc::Docker::Container depends_on: [database] properties: image: wordpress links: db: mysql port_bindings: 80/tcp: [{"HostPort": "80"}] docker_endpoint: str_replace: template: http://host:2345/ params: host: {get_attr: [docker_host, networks, private, 0]} outputs: url: description: Public address of the web site value: str_replace: template: http://host/wordpress params: host: {get_attr: [docker_host, networks, private, 0]}
You can deploy the template simply doing calling stack-create:
heat stack-create -f docker_sd.yaml -P image=fedora-software-config -P instance_type=m1.large docker_stack
The first part of the template is about exposing a Nova server with a Docker API endpoint listening for commands. We then create a database container and a WordPress container that we link to it. Using the port_bindings configuration, we expose the container on the Docker host. The URL output gives the private address where the service should be up once the stack is deployed.
Another alternative to software deployments is to simply use wait conditions. While less powerful, it can solve our problem easily; the corresponding template is shown at [6].
Being a first shot at solving that problem, there are several ways it can be improved. The main thing is that while you have a per-tenant Docker endpoint, it’s open without any authentication. It would be nice to use client certicates as shown at [7]. Then, for reproducibility, having an image with Docker installed and configured properly would simplify the template quite a bit.
We haven’t got the final word on how Docker and containers will integrate in OpenStack. There is a driver for Nova [8] actively developed, but it’s unclear (at least to me) if it fits Nova model or not, and it certainly doesn’t expose all features that Docker has to offer. I expect something like a container service to emerge, but in the mean time using Heat gives you some nice capabilities.
[1] http://docs.openstack.org/developer/heat/template_guide/contrib.html#dockerinc-resource
[2] https://docs.docker.com/reference/api/docker_remote_api/
[3] https://review.openstack.org/106120
[4] https://github.com/openstack/heat/blob/master/contrib/docker/docker/README.md
[5] https://github.com/openstack/heat-templates/blob/master/hot/software-config/elements/README.rst
[6] https://gist.github.com/therve/0e1148296c6c9b43cb55
[7] https://docs.docker.com/articles/https/
[8] https://github.com/stackforge/nova-docker
[…] folks at eNovance have a write-up on multi-tenant Docker with OpenStack Heat. It’s an interesting write-up, but not for the faint of heart—to make their example […]
[…] to the target environment of choice. OpenStack Orchestration (Heat) declared support for Docker orchestration starting from the Icehouse release. Cloudify is an open source TOSCA based […]
[…] engine to the target environment of choice. OpenStack Orchestration (Heat) declared support for Docker orchestration starting from the Icehouse release. Cloudify is an open source TOSCA based orchestration that works […]
[…] engine to the target environment of choice. OpenStack Orchestration (Heat) declared support for Docker orchestration starting from the Icehouse release. Cloudify is an open source TOSCA based orchestration that works […]