Crossplane – Let your Kubernetes to build your cloud infrastructure

Recent days infrastructure automation is one of the most focused across most of the organization, why? One of the main reasons is cloud adaptation. So, with that they can manage the complete infrastructure on AWS, Azure, GCE and other cloud providers.

As there are more tools to manage the infrastructure, it will be hard to manage and learn more tools for your SRE or DevOps team. What if you can manage your infrastructure via Kubernetes? Yes, you read correct. In this case, you are going to love Crossplane for sure. Let’s see what is crossplane and how to use it, also will try to compare with Terraform.

What is Crossplane?

Crossplane is a tool created by Upbound which has been released in December of 2018. It has been accepted as an incubating project by the CNCF (Cloud Native Computing Foundation) in 2020.

Crossplane is developed as a Kubernetes add-on and extends any Kubernetes cluster with the flexibility to provision and manage cloud infrastructure, services, and applications. Crossplane uses Kubernetes-styled declarative and API-driven configuration and management of infrastructure, on-premises or within the cloud.

Crossplane can be considered as a Kubernetes add-on, which means that it makes use of custom resources to provide all of its functionality. There are 4 kinds of resources,

Providers

• They are the first kind of package in Crossplane’s terminology
• A package is simply an OCI image, like a Docker image.
• It installs CustomResourceDefinitions to allow for the provisioning of resources on an external service like a cloud provider
• As of today, providers exist for AWS, GCP, Azure, ArgoCD, GitLab, Github, Kubernetes, and many more

Managed resources

• They are installed by Providers
• They represent infrastructure resources

Configurations

• They are the second kind of package according to Crossplane.
• They leverage the CompositeResourceDefinition and Composition features of Crossplane.

Composite resources

• They are defined using Crossplane configuration.
• They group-managed resources together to allow for the creation of more complex, business-oriented infrastructure resources.

Why Crossplane?

• A Developer-Friendly API:
  As Crossplane builds upon Kubernetes, a developer creates a resource to request a specific service. For Kubernetes developers, this is straightforward.
• Powerful:
  Crossplane includes support for AWS, Azure, GCP, and Alibaba, and also the community is adding support for several providers.
• Production-Ready with the assistance of K8s:
  It uses Kubernetes controllers and additionally the idea of continuous reconciliation to run the platform. If some resource breaks (which it will), Crossplane will examine and change the resource state to desired configuration.

Benefits of Crossplane?

Here we have captured some benefits of using Crossplane in your infrastructure.

GitOps

As we have seen, GitOps may be a future, with that considering and Using Crossplane, we able to implement GitOps best practices not only on the resources running within our Kubernetes cluster but also on our infrastructure resources. Thus, all of our resources are kept in Git. An agent running within our cluster will be provided with access to our Git repository to pull any changes within Git into our Cluster.

This allows us to set up advanced deployment mechanisms. Whenever any changes occur, whether on our resources or our infrastructure, the agent will recognize those changes and pull them automatically into our cluster; should they pass our checks.

Continuous Reconciliation

Crossplane helps that we can declaratively manage our infrastructure. Meaning, we will specify within our code the properties of our infrastructure resources and Crossplane will ensure those are created as defined. The code, which details our infrastructure, is referred to as our desired state and the resources that Crossplane will create become the actual state of our infrastructure.

Once the resources have been created, Crossplane will continuously monitor the state of our infrastructure resources. Which makes sure the actual state same as desired state if not, Crossplane will then try to modify the actual state to match the desired state.

Consistency

As Crossplane allows us to define our infrastructure resources in Git. One way maintaining our infrastructure in code makes it possible to easily can manage and share across. Instead of allowing someone to make any manual changes, we can just share the files with other teams to create a new Kubernetes Cluster or similar. As a result, our infrastructure can become more consistent across teams.

Shifting Left

Shifting Left is the process of empowering developers by ensuring they are equipped to participate in infrastructure management and deployments. Instead of a dedicated DevOps team being responsible for creating and managing infrastructure in response to requests by the engineering teams, the engineering teams are provided with the necessary tools to address most common use cases themselves.

Using Crossplane, the DevOps team can define what type of infrastructure resources can be created and then provide the engineering team with examples of those resources. In contrast, if engineers have to have access to a cloud provider account to create resources, they may have less direction on which resources to create, how to create them, and how to manage the system most efficiently.

Thus, Crossplane makes it easier to empower engineering teams across the organization.

Installation of Crossplane

Let’s take a look at how Crossplane allows us to provision cloud resources.

Prerequisites

• Get a Kubernetes Cluster or Kind Cluster or Minikube v0.28+
• Helm, minimum version v3.0.0+.

Installing Crossplane

Use Helm 3 to install the latest official stable release of Crossplane.

# kubectl create namespace crossplane

# helm repo add crossplane-stable https://charts.crossplane.io/stable

# helm repo update

# helm install crossplane --namespace crossplane crossplane-stable/crossplane

Check Crossplane Status.

# helm list -n crossplane

# kubectl get all -n crossplane

Install Crossplane Command Line Interface (CLI)

Use the following command to install crossplane CLI

# curl -sL https://raw.githubusercontent.com/crossplane/crossplane/release-1.5/install.sh | sh

Move executable to path /usr/local/bin

# mv kubectl-crossplane /usr/local/bin

Verify the installation.

# kubectl-crossplane --version

Terraform vs Crossplane

If you are more bound with terraform or spend more time on learning it, you can ask, why should I use Crossplane, as I can manage everything with Terraform. Well, that’s valid question, as both the tools are doing the same at last, provision infrastructure as a code. Still, there is too long to go with cloud, as we are still on earlier stage on cloud, the future may be very huge. So, it is important with pick the best one or two and see where we can use which one. So, Let’s understand by analyzing a few key differences between these tools.

YAML or HCL?

The first difference between both tools is the language use to operate them. As Crossplane is Kubernetes native, it uses YAML to distribute its features. Terraform has its own programming language, HCL, which is something to learn before you can start using it efficiently.

By using YAML, Crossplane can be used in conjunction with powerful templating technologies such as Helm. It’s also a much simpler language to read than HCL. HCL, being a very specific language built only for Hashicorp tools.

State management

A big part of Terraform is state management. The state in Terraform terms represents the mapping between your code and the real cloud infrastructure your code is supposed to create. It’s used to compute the differences between both of them and to know whether to make changes or not. Materially speaking, the state is a JSON file that you store locally or remotely.

With Crossplane, the state of your infrastructure is represented by all the Kubernetes resources you created. To consult your Crossplane state, you can just query the Kubernetes API using kubectl.

Reconciliation loop

A reconciliation loop can be explained using this simple schema, as we need in benefits of crossplane.

It’s a very common concept in Kubernetes because it’s how operators work. Crossplane can be considered like an operator as it relies on Custom Resources to provide its functionality. The way that it’s materialized is that Crossplane regularly monitors its resources to watch for changes. For example, if you change a parameter on a managed resource, Crossplane will pick the change up and carry out the said change in the real infrastructure.

This is a key difference with Terraform, which works in a more asynchronous way. The only time when terraform will reconcile the world with its state is when running the terraform apply command. It will then compute all the differences between both worlds and let the user choose whether to apply those changes to the existing infrastructure or not.

Immutable fields management

Whenever you start to manage cloud infrastructure, you will quickly learn about immutable parameters. For example, the name of an AWS VPC is an immutable field, which means that it cannot be changed during the lifecycle of this resource. With Terraform, if you created a VPC with your code, and then decide to change its name, terraform will suggest you replace completely the resource by destroying the former VPC and creating a new one with the same characteristics, except for the name.

Crossplane functions the opposite way. When an immutable property of a managed resource is changed, Crossplane will never take it upon itself to destroy the resource and recreate it with the new property. This is in part due to the fact that Kubernetes does not support the definition of immutable fields for custom resources.

GitOps approach

You could guess already after understanding the different, Crossplane is a perfect match with GitOps practices. If you already manage your Kubernetes resources using a tool like ArgoCD or FluxCD, or if you are following our ArgoCD series, managing Crossplane and its dependent manifests should be very easy. Simply pushing commits on a git repository will conduct changes on your external infrastructure with such a setup. It’s really convenient and also how working on infrastructure as a team should be.

Implementing GitOps using Terraform is a tedious because you have to rely on some sort of CI/CD machinery like a custom pipeline or tools such as Atlantis.

Private resources management

One of the first things we’ve realized with Crossplane is that it makes the management of isolated resources (in network terms) easy. For example, let’s say that you need to provision a database for your application. You can easily create an azure postgres instance with either Terraform or Crossplane, but you then need to add some users and create some databases to make it usable by your backend application.

This postgresinstance should be in a private network to guarantee that it’s not accessible from the internet. Realistically, your Kubernetes cluster should be able to access the network of your RDS instance because your backend pods need to use it. With this, Crossplane should be able to access the postgresinstance via the network out of the box. With a postgres provider you can create the users and databases that your application needs.

To do such a thing with Terraform, you have to set up your network connection on your computer so that it can access private resources (with an SSH tunnel for example). Another solution would be to have your Terraform code running into a network connected to your database network, which is a big task in itself.

Infrastructure overview

Using Terraform, having an overview of your infrastructure is not easy. You cannot really be sure that all of your code matches your real-world infrastructure except when running plans. By using Crossplane, you leverage the reconciliation loop and availability of the Kubernetes API. You can check at all times if your infrastructure is in sync with your desired state.

With ArgoCD, it’s even easier to see the entire state of your infrastructure via its UI. With this, you can check whether your infrastructure is healthy or not. This is a big positive compared to the effort needed to do such a check with Terraform.

Provider

Crossplane is still a new platform in this technology, all of its features are pretty cutting edge, including the providers. If you compare it with Terraform, Crossplane is in early as even there are providers, still they are not matured to provide the feature some organization looking for. As Terraform was early player on this, they have learned lot and added more feature with their definitions. For example, for now crossplane limited to create the DB instances where terraform can add advance features like read replicas.

Having said that the Crossplane team is working on is the terrajet providers to address such a gap. It’s a tool that can process Terraform providers to generate Crossplane providers. But still, this is in earlier state, we still have to look forward and see how it is going to get matured.

For once, you cannot link resources between providers. For example, you may have all your infrastructure built with the classic AWS provider. If a resource is not in this provider but can be created using the terrajet provider, you won’t be able to link them together

Secondly, installing a terrajet provider can greatly hinder the performance of your Kubernetes APIServer. For example, the AWS terrajet provider installs more than 763 CRDs (about 160 for the classic one). That has the effect of slowing down any requests done to Kubernetes by a lot (a few seconds to dozens of seconds depending on what your cluster is running). An issue is open on the topic of adding a feature to only install a portion of the CRDs bundled by the provider.

Kubernetes ⇒ Single Point of Failure

Be aware of using Crossplane, you need to make sure that your Kubernetes cluster is in working state all the times. If there is any issue with your cluster, you won’t be able to make any changes to your infrastructure without running the risk of having a difference between the Crossplane state and the real world.

Also, user management, should be very well planned and granted. Example like delete resource permission should be crafted very well that may cause an issue in the real-world. Also, please should plan for how will you manage the Kubernetes cluster on which Crossplane runs? Would you be brave enough to manage it with Crossplane itself after provisioning it? Or will you leave some written documentation with manual steps or Terraform code to bootstrap the needed cloud infrastructure? This could be big open topic to discuss. So Terraform may looks like unavoidable, but lets keep eye on the crossplane, so will see does it acquire the world like terraform or will be a yet another tool in the market.

Conclusion

Our thought about Crossplane is as this is still on earlier state, we may need to see in future, this could be one of the unavoidable tool in future, if there is wide support from multiple providers, as other tools are too matured and handling very complex requirements.

Check our ArgoCD series on https://foxutech.com/category/kubernetes/argocd/