Operating systems like CoreOS use Docker to run applications on top of their lightweight platform. Docker in its turn provides utilities around technologies like Linux container technology (e.g. LXC, systemd-nspawn, libvirt). Previously Docker could be described as the “automated LXC”, now it’s actually even more powerful. What it definitely is simplifying and enhancing the possibilities of container technology.
Continuous delivery
Rolling out containers is quick and very easy. It helps companies to improve development work flows. Developers can perform testing and deploy their applications much easier than before. Additionally, the use of Docker enhances the process from development, up to running software in production. This is achieved by using smaller units, which are easier to create, monitor and secure.
Supporting multiple technologies
Docker uses the possibilities of the hypervisor management tooling like libvirt and systemd-nspawn. It has an ongoing development and supports more and more features, to simplify the management of containers. With each of the component getting more stable, the base of Docker reached a level of stability, use it in production.
Enhancing security
With the right measures, Docker will also enhance security. For example due to running (smaller) individual units, controlling them is easier. One benefit of small containers is providing application owners and administrators a better insight what software, protocols and network flows are needed for individual services.
IT architects and security professionals will definitely benefit from container technology as well. Architects gain fine grained building blocks, to define new services and enhance existing ones. Security professionals benefit from a better segmentation, and minimizing the permissions needed in each individual container.
Docker Security and Risks
Software packages can solve existing security threats, or actually introduce new risks. This is also the case when using Docker. While it can help in reducing risks by using compartmentalization, the implementation might have its flaws.
Risks
One common threat of new services is a low(er) stability, which forms a risk to the availability of a service. Another one is information disclosure, as the service might be lacking appropriate controls. Usually new technologies have a need of adding new features quickly, which might result in sloppy programming. Often this will result in software vulnerabilities, including ones which are security related.
Unfortunately Docker already had its share of security vulnerabilities, but they took a more active stance to improve the security of their products.
Methods and best practices
To reduce the risks when using fairly new technologies, we will have a look at the methods available to Docker. In particular, we look how Docker can increase security. After that, we provide some best practices when dealing with Docker containers.
Maturity level
Container management is a fairly new technology, which leaves many professionals with a knowledge gap in this area. Additionally, not many people are capable (yet) of making a proper security assessment. For example, auditors might not be able to ask the right questions regarding the implementation of containers.
This mature level risk also includes lacking technical auditing of containers, or difficulties in maintaining proper and up-to-date documentation. After all, the flexibility containers provide also means containers can run more easily on a different system, at any given time. This might need another level of documentation, to reflect the possibilities of each individual unit.
Containers do not (fully) contain
While containers are used to compartmentalize and limit resources, they actually don’t fully contain. For example a process running in the container with UID 1000, will have the privileges of UID 1000 on the underlying host as well. Along these lines, a process running as root (UID 0) in a container has root-level privileges on the underlying host when interacting with the kernel.
This risk will be soon be mitigated, when user namespaces are fully being implemented. The first step is already made in the form of subordinate user IDs (subuid). This helps mapping existing user IDs on the host system, into different user IDs within each container.
Security Benefits of Docker
Segregation of applications
Normally applications all run on the same host system. By using container technology we can segregate them, making it easier to determining traffic flows.
Flexible attitude
With containers being smaller individual units, they become dynamic, or flexible. The work flow to maintain them is more flexible as well. Great for security patching, testing and releasing the updated containers into production.
Focus on automation
Docker has a clear focus on automation. They have supporting products like Docker Machine, Swarm (clustering) and Compose, to simplify management of many containers.
Limiting information disclosure
Containers can limit resources assigned. This helps us in limiting the amount of information available to the system (and an evil attacker). Each container gets the following components:
- Network stack
- Process space
- File system instances
Limiting resources are achieved by using namespaces. Namespaces are like a “view”, which only shows a subset of all resources on the system. This provides a form of isolation: processes running within a container cannot see, or affect, processes in other containers, or the host system itself.
Protection Methods
By using Docker properly, some of its defenses can be used. Unfortunately the tooling does not actively help yet to leverage all possibilities. You are the one to properly configure, use and update Docker. Our hope is that Docker in the future will be stricter with some options, or at least advise the user to some extent.
Limited capabilities
Linux has support for “capabilities”, which can be seen as roles. A role could be opening a network socket, to craft a packet and put it onto the wire. Normally these kind of roles are only available to the root user. By splitting them in the form of capabilities, they can also be assigned to individual processes as well. This way a piece of software can still open up a socket (with “root permissions”), while not being able to load a new kernel driver.
Containers will run with a limited capability set. So even if someone breaks into the container, the host system is to some extent protected.
Examples:
- Mounting operations
- Access to raw sockets (prevent opening privileged ports, spoofing)
- Some file system operations (mkdev, chown, chattrs)
- Loading kernel modules
The configuration and usage of capabilities will be covered later. For now it is good to know that Docker by default drops a list of capabilities:
- CAP_AUDIT_WRITE = Audit log write access
- CAP_AUDIT_CONTROL = Configure Linux Audit subsystem
- CAP_MAC_OVERRIDE = Override kernel MAC policy
- CAP_MAC_ADMIN = Configure kernel MAC policy
- CAP_NET_ADMIN = Configure networking
- CAP_SETPCAP = Process capabilities
- CAP_SYS_MODULE = Insert and remove kernel modules
- CAP_SYS_NICE = Priority of processes
- CAP_SYS_PACCT = Process accounting
- CAP_SYS_RAWIO = Modify kernel memory
- CAP_SYS_RESOURCE = Resource Limits
- CAP_SYS_TIME = System clock alteration
- CAP_SYS_TTY_CONFIG = Configure tty devices
- CAP_SYSLOG = Kernel syslogging (printk)
- CAP_SYS_ADMIN = All others
Usage of seccomp
Secure Computing, or seccomp, helps with the creation of sandboxes. It does so by defining what system calls should be blocked. The latest version of seccomp provides this syscall filtering by using the Berkeley Packet Filter (BPF), previously used for filtering network traffic.
Containers currently have the following syscalls disabled (since LXC 1.0.5):
- kexec_load
- open_by_handle_at
- init_module
- finit_module
- delete_module
System calls blocked by default in Linux containers
When any of the blocked syscalls is made, the kernel will send a SIGKILL signal to stop the related process.
Digital Signature Verification
Starting with Docker version 1.3.0 all images are verified after downloading. This is a great step in enhancing the level of trust for downloads. This is why we have done this for our auditing tool Lynis as well. Like our website, Docker is also providing their website HTTPS-only. Another level of trust and ensuring you are are the right place, downloading the tools from Docker.
Current issues with Docker
Root permissions
Right now there is a small issue left with Docker, which is the requirement of running the daemon with root permissions. Docker is aware of it and plans to define well-audited sub-processes, which do not longer require root permissions. Additionally, each sub-process will run with a very limited scope, increasing the security level of each component and enhancing stability.
Lack of full User namespace implementation
Currently there is still no full user namespace implementation. Something which is out of the control of Docker. When the LXC userland tools are evolved and include the support, Docker can leverage the possibilities. The first actions like user mapping is available, so full support is expected soon.
User 0 in container = User 0 on host
One of the risks due to the missing User namespaces, is that the mapping of users from the host to the containers is still an one-to-one mapping. Previously user 0 in the container was equal to 0 on the host. In other words, if you container is compromised, it doesn’t take much to compromise the full host. Fortunately this is work in progress. LXC already supports a mapping option, to map user ID 0 in the container to another (high) ID on the host.
Default allow all
By default all IP traffic is allowed between containers. This means they can ping each other, but also send other forms of traffic. It would have been better if Docker applied a “deny all by default” principle. This forces the maintainer of the container to think about what kind of traffic is needed between individual containers.
Fortunately traffic can be filtered and is absolute advised for systems in production.
Best Practices
With all these risks and possibilities, let’s extract some of the lessons. These best practices help you create more safe services and enhance the security of existing containers.
Do not run software as root
This tip might sound to simple, but still many developers run their software as the root user. Containers still cannot contain properly, which might result in a full host compromise if a container is compromised. Therefore, run your software packages like you would run them on a normal host.
Use Docker in combination with AppArmor/SELinux/TOMOYO
Ubuntu comes with ready-to-use AppArmor templates for LXC. It is always a good thing to know what your software does. This includes knowing what paths and permissions your software components need to function properly. With this information each piece can be restricted to the bare minimum needed, preventing permission escalation and unauthorized information disclosure (or worse).
To achieve the right policies, make sure to monitoring your applications from the start, including the related framework you are using. Each of them provides the means to monitor, so use them.
Within the container configuration the related AppArmor profile can be defined with lxc.aa_profile.
Use seccomp to limit syscalls
Support for seccomp is available to (at least) CentOS, Debian, Fedora, Gentoo, Oracle, Plamo and Ubuntu. You can use seccomp by altering the container configuration and define the seccomp rule set to be used:
lxc.seccomp = /usr/share/lxc/config/common.seccomp
For Docker this functionality can be activated by using the –lxc-conf parameter to docker run.
LXC configuration option: lxc.seccomp
Limit traffic with iptables
By default all containers use the docker0 interface as a bridge. Like on a normal host you can limit traffic, to block unauthorized traffic streams.
For full details we suggest to read the advanced networking blog post at Docker.
GRSEC and PaX
When possible, use a hardened Linux kernel, with kernel patches. Grsecurity and PaX are two examples which help in hardening the host system.
Using user mappings
To counter the issue that user 0 in a particular container equals root on the host system, LXC allows you to remap user and group IDs. The configuration file entry would look something like this:
lxc.id_map = u 0 100000 65536
lxc.id_map = g 0 100000 65536
This maps the first 65536 user and group IDs within the container to 100000-165536 on the host. The related files on the host are /etc/subuid and /etc/subgid. This mapping technique is named subordinate IDs, hence the “sub” prefixes.
For Docker this means adding it as a -lxc-conf parameter to docker run:
# docker run -lxc-conf=”lxc.id_map = u 0 100000 65536″ -lxc-conf=”lxc.id_map = g 0 100000 65536″
Use –cap-drop and –cap-add
(Since Docker 1.2.0)
With the earlier described Linux capabilities we can tell Docker what specific roles should be given to a container Actually, we can use both the add and drop option together. By first allowing all, then dropping some capabilities, we can limit the capabilities. The short version is just dropping permissions. The alternative is just adding the related capabilities, which more resembles the “deny all” principle.
# docker run –cap-drop=CHOWN
# docker run –cap-add=ALL –cap-drop=MKNOD
A safe list of capabilities to drop are:
- audit_control
- audit_write
- mac_admin
- mac_override
- mknod
- setfcap setpcap
- sys_admin
- sys_boot
- sys_module
- sys_nice
- sys_pacct
- sys_rawio
- sys_resource
- sys_time
- sys_tty_config
See the capabilities (7) man page for all details about these capabilities.
LXC configuration options: lxc.cap.drop and lxc.cap.keep
Do not run SSH in containers
Use “docker exec -it mycontainer bash” instead to manage your containers.
Do not run –privileged on containers
(Since Docker 1.3.0)
For containers which already have SELinux/AppArmor support, use –security-opts instead. This gives it the appropriate security profile, instead of giving away too much permission within the container.
# docker run –security-opt label:type:svirt_apache -i -t centos \ bash
Related options for SELinux:
- –security-opt=”label:user:USER” (set label user)
- –security-opt=”label:role:ROLE” (set label role)
- –security-opt=”label:type:TYPE” (set label type)
- –security-opt=”label:level:LEVEL” (set label level)
- –security-opt=”label:disable” (disable label confinement completely)
Options for AppArmor:
- –secutity-opt=”apparmor:PROFILE” (set AppArmor profile)
For more options have a look at the Docker Run Reference.
Upgrade your Docker version on a regular basis
Most software packages have bugs, small programming errors. With Docker also being under heavy development, bugs are solved and new features added. Regular updating and making Docker part of your software patch management process, is advised.
Secure Docker client connections
Set the DOCKER_HOST and DOCKER_TLS_VERIFY variable, to use TLS for connecting to Docker instances. See https://docs.docker.com/articles/https/ for detailed instructions on setting this up
.){kind=link}