In this post will see what is Kubernetes hardening and how we can use Kubescape for Kubernetes hardening.
We are seeing Kubernetes growth has exceptional, over the years across all the platform and environments with CI/CD pipeline etc. Also most of us moving to microservices application model, and we are just focusing on migrating to Kubernetes, how many of us considering the security of the Kubernetes cluster? Well, yes there is few, but is that sufficient? Answer could be no or need more improvement or we may planned to after migration. But skipping at initial time, it could be huge risk, As per Red Hat recently reported, human error is a leading cause of Kubernetes security mishaps. Indeed, 94% of those surveyed admitted they have experienced a Kubernetes and container environments security incident in last one year. Worse still, more than half of respondents, 55%, ended up delaying Kubernetes application production developments due to security issues in last one year. So if you have best system in-place you could avoid atleast the human errors.
Likewise, infrastructure security, We may lot of restriction implemented across the hosting environments, but that won’t enough, because mostly those could be compromised, if there is any Human error or any malicious software/package or other possible reason which we skipped at initial time of setup. Because, as like we are moving Kubernetes, hackers also could start target the Kubernetes environments for data theft, denial of service, or cryptocurrency mining, etc.
So if the Kubernetes is your organizations future, it is our responsibility to secure the Kubernetes cluster, for that we should implement techniques that harden the Kubernetes security. In this article, will understand the Kubernetes hardening and how open-source tool like kubescape can help us.
What is Kubernetes Hardening?
Kubernetes hardening involves security measures taken to secure Kubernetes systems. Here are some recommendations from NSA.
- Scan containers and Pods for vulnerabilities or misconfigurations.
- Run containers and Pods with the least privileges possible.
- Use network separation to control the amount of damage a compromise can cause.
- Use firewalls to limit unneeded network connectivity and use encryption to protect confidentiality.
- Use strong authentication and authorization to limit user and administrator access as well as to limit the attack surface.
- Capture and monitor audit logs so that administrators can be alerted to potential malicious activity.
- Periodically review all Kubernetes settings and use vulnerability scans to ensure risks are appropriately accounted for and security patches are applied.
These could be helpful, but mostly this all generic. As there is lot of hard part should be addressed. For example, we all know we shouldn’t run applications as root, but we could see, many Kubernetes container services run as the root user, and applications execute within them as root even though they don’t need privileged execution. Even there are warning, developers build container applications that execute as root. Why? Because it’s so easy. Same time it also dangerous.
And, of course, even Kubernetes has its fair share of its own security problems. For instance, the Cybersecurity and Infrastructure Security Agency (CISA), NSA’s partner in this guide, warned of a critical, with a terribly high CVSS severity score of 8.8, Kubernetes Capsule Operator reverse proxy privilege escalation flaw, CVE-2022-23652.
As a DevOps engineer/SRE should adhere to the recommendations in the Cybersecurity Technical Report to ensure the security of applications and keep their systems up to date with patches, upgrades, and updates to reduce risk. To make sure that the proper risks are considered, and security patches are applied, the NSA and CISA also advise conducting regular reviews of Kubernetes settings and vulnerability scans.
As the default install of Kubernetes isn’t necessarily secure. For example, the network access to the control plane may be too permissive or the admission controller policies may allow dangerous images to run. Your orchestration platforms must be set up in a secure way with proper configurations and be periodically validated so that they haven’t drifted over time. Hardening can reduce risk by shrinking the attack surface and consequently making an attacker must spend much more time to accomplish their goal.
As like common release risk or platform risk, even Kubernetes could get compromise due to Supply chain risks, malicious package/software(like log4j vulnerability), and human error. As Risks in the supply chain are frequently challenging to eliminate and may appear during the infrastructure acquisition process or container build cycle.
Parts of the Kubernetes architecture, such as the control plane, worker nodes, or containerized services, can have flaws and incorrect configurations that threat actors with malicious intent can exploit. Human error can be from administrators, users, or cloud service providers. A company’s Kubernetes infrastructure may be vulnerable to attack from insiders with special access. Like Container escapes are not uncommon, as a recent Linux kernel vulnerability (CVE-2022-0185) shows. Misconfigurations can also allow an attacker to pivot to the node OS.
USA’s National Security Agency (NSA) and the Cybersecurity and Infrastructure Security Agency (CISA) released, “Kubernetes Hardening Guidance“. The guidance details threats to Kubernetes environments and provides secure configuration guidance to minimize risk.
If you are managing/planning to have large infrastructure or applications, it could be hard to follow all the instruction mentioned in the guidance, or it will be so risk identifying the issue at right time with manual effect or writing some custom validation scripts etc. Instead building own system, we can make use of the tools like Kubescape, which extremely valuable if you want to accomplish the highest level of application security with the least amount of development effort.
Kubescape
Kubescape is a K8s open-source tool providing a Kubernetes single pane of glass, including risk analysis, security compliance, RBAC visualizer, and image vulnerability scanning.
Kubescape scans K8s clusters, YAML files, and HELM charts, detecting misconfigurations according to multiple frameworks (such as the NSA-CISA, MITRE ATT&CK®), software vulnerabilities, and RBAC (role-based-access-control) violations at early stages of the CI/CD pipeline, calculates risk score instantly and shows risk trends over time.
It has become one of the fastest-growing Kubernetes tools among developers due to its easy-to-use CLI interface, flexible output formats, and automated scanning capabilities, saving Kubernetes users and admins precious time, effort, and resources.
Kubescape integrates natively with other DevOps tools, including Jenkins, CircleCI, Github workflows, Prometheus, and Slack, and supports multi-cloud K8s deployments like EKS, GKE, and AKS.
How does Kubescape work?
Kubescape is based on OPA engine and ARMO’s posture controls. it retrieves the Kubernetes objects from the API server and scans them by running a set of regos snippets developed by ARMO. It establishes how well your Kubernetes configurations meet the best practice recommendations from the different frameworks included.
The output results are printed in a “console friendly” manner by default, but they can be also retrieved in JSON or Junit format for further processing.
Kubescape is running the following tests according to what is defined Kubernetes Hardening Guidance by NSA and CISA.
- Non-root containers
- Immutable container filesystem
- Privileged containers
- hostPID, hostIPC privileges
- hostNetwork access
- allowedHostPaths field
- Protecting pod service account tokens
- resource policies
- control plane hardening
- Exposed dashboard
- Allow privilege escalation
- Applications credentials in configuration files
- Cluster-admin binding
- Exec into containers
- Dangerous capabilities
- Insecure capabilities
- Linux hardening
- Ingress and Egress blocked
- Container hostPort
- Network policies
- Symlink Exchange Can Allow Host Filesystem Access
- Anonymous Requests
These tests performed by connecting to the Kubernetes API and using the OPA engine, according to the guide determined by NSA and CISA.
Installation
Prerequisites
You need to have installed Kubectl and have a running cluster before starting this tutorial.
How to Install Kubescape
The Kubescape is available on Windows, macOS, and Linux. This tutorial will give cover install on Windows, Linux, and MacOS.
Installing Kubescape CLI on Windows
Use the following command to install Kubescape CLI on Windows using PowerShell:
> iwr -useb https://raw.githubusercontent.com/kubescape/kubescape/master/install.ps1 | iex
If you have downloaded Kubescape successfully without errors, you will get the following output:
There is a possibility that the installation process might fail because PowerShell has not been enabled or you can get an error that says, “the internet explorer engine is not available”. To resolve the latter error, go ahead and download internet explorer. After you have installed the internet explorer start the Kubescape installation procedure; this time it will install successfully.
Also, change the execution policy and enable PowerShell using the following command if you had an installation error:
$ Set-ExecutionPolicy RemoteSigned -scope CurrentUser
Installing Kubescape CLI on macOS
Use the following two commands to install Kubescape using Homebrew. If you don’t have Homebrew, go ahead, and download it from brew. 1. Start by adding the Kubescape GitHub repository:
$ brew tap kubescape/tap
Install Kubescape:
$ brew install kubescape-cli
Installing Kubescape CLI on Linux
Go ahead and download Kubescape CLI binary files from GitHub. For now, Kubescape has only Ubuntu binary files.
Download the zip file and then unzip to install the package,
# wget https://github.com/kubescape/kubescape/archive/refs/tags/v2.0.171.zip
# unzip v2.0.171.zip
# cd kubescape-2.0.171/
# ./install.sh
Successful Installation Confirmation and Using Kubescape -
h Command
Use the following to check the Kubescape version:
# kubescape version
Use the following command to get all the available commands:
# kubescape -h
You can also get extra information on a certain command by adding the -h flag. For example:
# kubescape scan -h
Limitation:
Cluster scanning
Kubescape reads cluster information from the cloud vendors’ API. There is a list of authorization required by the entity (see the SDK of the given vendor). Kubescape uses SDKs to retrieve this information. This authentication information must be available in the running environment of Kubescape
- AWS IAM roles for EKS access, see more here Amazon Web Services integration
- GCP IAM roles for GKE access, see more here Google Cloud Services integration
- Azure AKS is not yet supported
- DigitalOcean API is not yet supported
Vulnerability scanning
Private repositories are supported via:
- ImagePullSecrets (in podSpec or serviceAccount)
- AWS IAM roles for ECR access, see more here Amazon Web Services integration
- GCP IAM roles for GCR access, see more here Google Cloud Services integration
- Azure ACR authentication is not yet supported
Here you can check how we can scan kubernetes resources using kubescape: https://foxutech.com/how-to-scan-kubernetes-resources-using-kubescape/