How to Autoscale Pods in Kubernetes Based on HTTP Requests

In our previous article, we have learned about autoscaling, in this will understand more scenario like how to autoscale the pods in Kubernetes based on HTTP Requests.

Autoscaling Pods is a fundamental feature in Kubernetes to adjust the resource limits based on the usage or changing workload of Pods to maintain availability of the applications. As we are aware, kubernetes provides two types of autoscaling: Horizontal Pods Autoscaling (HPA) and Vertical Pods Autoscaling (VPA).

We have understood how the autoscaling works with different resource metrics, in this Scaling up Pods based on HTTP requests requires custom metrics or external metrics. Kubernetes does not natively provide to auto scale resources based on http requests. To achieve this, we need external metrics. Horizontal Pods Autoscaling (HPA) supports the external metrics, based on the it scales the resource dynamically.

1. Let’s Deploy a sample application

apiVersion: apps/v1
kind: Deployment
metadata:
  name: testapp-backend
spec:
  selector:
    matchLabels:
      app: testapp-backend
  template:
    metadata:
      labels:
        app: testapp-backend
    spec:
      containers:
      - name: testapp-back
        image: registry.k8s.io/hpa-example
        ports:
        - containerPort: 80

2. To autoscale our application based on HTTP request we need to capture the metrics of our HTTP requests from our application. For example, for .net applications we can use Prometheus and instrumenting your application using appropriate metrics.

using prometheus; 
 
var httpRequestsCounter = 
Metrics.CreateCounter("http_request_total", "Total HTTP Requests");

3. Create a Horizontal Pod Autoscaling (HPA) external metric file or custom metric file and apply it to pod that we created to auto scale the resource based on HTTP requests.

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: testapp-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: testapp-backend
  minReplicas: 2
  maxReplicas: 10
  metrics:
- type: External
  external:
    metric:
      name: http_request_total
    target:
      type: AverageValue
      averageValue: 80     ##Adjust this value based on your requirements. 

To create the autoscaler, run below command on terminal

# kubectl create -f testapp-backend.yaml

horizontalpodautoscaler.autoscaling/backend-app-hpa configured

Note: external.metric.name should be same as the metric name you are using in your backend application.

Use the following command to monitor the HPA’s behavior

# kubectl get hpa backend-app-hpa

To get the description of HPA:
# kubectl describe hpa backend-app-hpa