Cloud application automated deployment using salt and docker

Cloud application automated deployment using salt and docker

Salt, or SaltStack, is a remote execution tool and configuration management system, based on the community-sourced Salt platform. The remote execution capabilities allow administrators to run commands on various machines in parallel with a flexible targeting system. The configuration management functionality establishes a client-server model to quickly, easily, and securely bring infrastructure components in line with a given policy.

A years ago, deploying and managing an application across multiple virtual machines using multiple public clouds was considered a very difficult use case. Companies like Google, FaceBook have divided the orchestration technology they have developed over the years to address scalability challenges. Today, using Docker containers in conjunction with Salt-Cloud , and some Salt state, we only need to take tens of minutes to process. And because we use SaltStack’s declarative configuration management, we can operate in the following modes in a real production environment.

salt cloud provisionAt the heart of this case is the deployment of one or more applications on one or more virtual hosts on one or more public cloud providers.

For this simple purpose, we will make the following Assumption on this case:

  • Assume that you are familiar with Salt
  • Assume familiar with Docker
  • Suppose you have installed a SaltSack Master.
  • Suppose you have done these things on a public cloud, using Digital Ocean (because adding a new cloud to Salt-Cloud is very simple)
  • Simulate real-world applications with a virtual apache service

salt cloud deploymentTo mimic a real-world application, you need to create an Apache Docker container. Conceptually, this container might be a front-end proxy, a web middleware service, a database, or other type of service that we need to deploy on production. For this, we create a DockerFile in a directory, build the container, and PUSH the container to the Docker repository. Here will see the example of digital ocean provision and deployment.

Install Salt and Salt Cloud via Bootstrap Script

The recommended way to install Salt Cloud is with a Salt Bootstrap script. This script will install Salt, Salt Cloud packages, and all required dependencies. Run the script with the -h flag to view the additional options available, or refer to Salt Bootstrap Guide for detailed instructions.

  1. Download the Salt Bootstrap script via curl:
# curl -o -L
  1. Execute the script and use the -L option to install Salt and Salt Cloud:
# sh -L

Step 1: Create a Dockerfile

# cat Dockerfile
FROM ubuntu:16.04
RUN apt-get update && apt-get install -y apache2 && apt-get clean && rm -rf /var/lib/apt/lists/*
ENV APACHE_LOG_DIR /var/log/apache2
CMD ["/usr/sbin/apache2", "-D", "FOREGROUND"]

Step 2: Build the container

# docker build -t motoskia/apache2 .

Step 3: Push the container

# docker push motoskia/apache2

When our application demo successfully PUSH to Docker’s repository, we are ready to coordinate its deployment. It is important to note in advance that we assume that you have installed a SaltStack, and if not, you can get the SaltStack Master installation based on this document . The next step is to configure Salt-Cloud on your chosen public cloud provider. Configuring Salt-Cloud is very simple, we need to create an SSH Key, which Salt-Cloud uses to install Salt Minion on newly created VMs, add this keypair to the public cloud provider, and use the certified API for our public The cloud creates a Salt-Cloud configuration file.

Step 4: Create an SSH Key Pair

Create the ssh key using following command follow the steps to setup providers

# ssh-keygen -t rsa
ssh-keygen# mkdir -p /etc/salt/pki/cloud

Next, create a file called do.pem inside the newly created directory:

# vim /etc/salt/pki/cloud/do.pem

Paste in the contents of your private key that is associated with one of your DigitalOcean keys you specified in the ssh_key_names directive in the provider file. You can usually get the contents of your private key by typing the following into your local computer:

# cat ~/.ssh/id_rsa

Paste that into the file at /etc/salt/pki/cloud/do.pem and then save and close the file.

In order for Salt to use the key, the file and directory containing it must have the correct permissions. Set the permissions by typing:

# chmod 700 /etc/salt/pki/cloud
# chmod 600 /etc/salt/pki/cloud/do.pem

Salt can now read the private key and use it to log into new servers.

Step 5: Upload SSH Key Pair

Login to your digital ocean account, and goto security section from left side menu. In that again go to security tab and scroll down and file the “Add SSH Key. Click that and add your .pub file.

The next step is to use the Digital Ocean’s API key to configure Salt-Cloud for our account, the size, geographic location, and image definition profile for the Digital Ocean virtual host, Salt-Cloud. certification file is saved in Salt-Master of /etc/salt/cloud.providers.d/.

Step 6: Configure Salt-Cloud

Create provider and profiles configuration

# cat /etc/salt/cloud.providers.d/digital_ocean.conf
  driver: digitalocean
# Digital Ocean personal access token create in digital ocean account page API section.
  personal_access_token: digitalocean_api_token
  ssh_key_file: /etc/salt/pki/cloud/do.pem
  ssh_key_names: salt-cloud
# cat /etc/salt/cloud.profiles.d/digital_ocean.conf
# Official distro images available for Arch, CentOS, Debian, Fedora, Ubuntu
  provider: do
  image: Ubuntu 16.04.4 x64
  size: 512MB
#  script: Optional Deploy Script Argument
  location: nyc3
  script: curl-bootstrap-git
  private_networking: True

  provider: do
  image: Ubuntu 16.04.4 x64
  size: 1GB
#  script: Optional Deploy Script Argument
  location: nyc3
  script: curl-bootstrap-git
  private_networking: True

  provider: do
  image: Ubuntu 16.04.4 x64
  size: 2GB
#  script: Optional Deploy Script Argument
  location: nyc3
  script: curl-bootstrap-git
  private_networking: True

It’s time to configure Salt for our app container. To do this, we need to create two Salt States , one for install Docker on a newly created VMs and one for the application container. Salt States is Salt’s declarative state configuration, which is executed by the salt-minion on the target host. States has a very rich set of features, and we tried to cover few of them.

Step 7: Create a Salt state for Docker

# cat /srv/salt/docker/init.sls
    - name: apt-key adv --keyserver hkp:// --recv-keys 58118E89F3A912897C070ADBF76221572C52609D
    - creates: /etc/apt/sources.list.d/docker.list
    - source: salt://docker.list
    - name: docker-engine
    - name: docker
    - require:
      - pkg: docker-engine

The first salt state defines the configuration for relying on and installing Docker

Step 8: Create Salt state for the application container

# cat /srv/salt/apache/init.sls
    - force: true
    - name: motoskia/apache2:latest
    - name: my_adpp
    - image: motoskia/apache2:latest
    - port_bindings: 80:80

This state first pull our image in the latest version then launch our container called my_app and with TCP port 80 exposed. Before we do this, let’s verify that the Salt Master is working properly. At this time, we know that at least one client should communicate with the Salt Master, which runs on the same server as the Salt Master.

Step 9: Verify that Salt is working

# salt '*'

Very satisfied, with the installation of Salt, everything worked orderly. Now we can use Salt-Cloud to prepare one of our container instances on our first virtual machine.

# salt-cloud --list-providers

You can test your API key by typing:

# salt-cloud --list-locations do

It should show you a list of the available regions for deployment.

Step 10: Provision a VM with an instance of the container

# salt-cloud --profile ubuntu_512MB_nyc3 onefox

When the run completes Salt-Cloud, it issues a YAML blob containing information about the newly created VM instance.

Once your instance is created you can run the highstates to install and run your docker application.

To get the particular minion ID run,

# salt-key -L

Pick anyone minion id from accepted key, and run. Here my minion id is “onefox”

# salt ‘onefox’state.highstate

Now you can see you application is up and running.