Skip to main content

KEDA Autoscaling with RabbitMQ

Introduction​

Autoscaling is crucial for efficiently managing application workloads in Kubernetes. It allows deployments to automatically scale up or down based on demand, optimizing resource utilization and ensuring application availability. KEDA (Kubernetes Event-Driven Autoscaler) enables scaling based on various event sources, including message queues like RabbitMQ. Using KEDA with RabbitMQ allows you to scale your deployments based on the number of messages in a queue, rather than relying solely on CPU or memory utilization. This approach is particularly useful for applications that process asynchronous tasks.

Prerequisites​

  • A running Kubernetes cluster.
  • kubectl installed and configured to connect to your cluster.
  • KEDA installed in your cluster. Refer to the KEDA documentation for installation instructions.
  • A running RabbitMQ instance accessible from your Kubernetes cluster.

Configuration​

To configure KEDA to autoscale your deployment based on the RabbitMQ queue length, you need to create a ScaledObject resource. Here's an example:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
meta
  name: rabbitmq-scaledobject
  namespace: your_k8s_namespace
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: your_k8s_deployment # The name of the deployment you want to scale
  pollingInterval: 10  # How often KEDA will check the queue length (in seconds)
  cooldownPeriod: 120 # How long to wait after the last scale event before scaling back down (in seconds)
  minReplicaCount: 1   # Minimum number of replicas
  maxReplicaCount: 10  # Maximum number of replicas
  triggers:
    - type: rabbitmq
      meta
        host: amqp://your_username:your_password@rabbitmq-test.rabbitmq-test:5672/your_vhost # RabbitMQ connection string
        queueName: your_rabbitmq_queue_name # The name of the RabbitMQ queue
        queueLength: "100" # Target queue length before scaling (e.g., scale up if the queue has more than 100 messages)

Explanation of Parameters:

  • metadata.name: The name of the ScaledObject.
  • metadata.namespace: The Kubernetes namespace where the ScaledObject will be deployed.
  • spec.scaleTargetRef.name: The name of the Kubernetes Deployment that KEDA will manage.
  • spec.pollingInterval: Defines how often KEDA will poll the RabbitMQ queue to check its length.
  • spec.cooldownPeriod: Specifies the cooldown period after a scaling event.
  • spec.minReplicaCount: The minimum number of replicas for the Deployment.
  • spec.maxReplicaCount: The maximum number of replicas for the Deployment.
  • triggers.type: Specifies the trigger type, which is rabbitmq in this case.
  • triggers.metadata.host: The connection string to your RabbitMQ instance. Important: Ensure this connection string is properly secured, especially the credentials. Consider using Kubernetes secrets to store sensitive information.
  • triggers.metadata.queueName: The name of the RabbitMQ queue that KEDA will monitor.
  • triggers.metadata.queueLength: The target queue length. When the number of messages in the queue exceeds this value, KEDA will trigger a scale-up event.

Deployment​

  1. Save the above YAML configuration to a file (e.g., rabbitmq-scaledobject.yaml).

  2. Apply the configuration to your Kubernetes cluster:

    kubectl apply -f rabbitmq-scaledobject.yaml -n your_k8s_namespace

Verification​

  1. Check the status of the ScaledObject:

    kubectl describe scaledobject rabbitmq-scaledobject -n your_k8s_namespace

    This command will show you the current status of the ScaledObject, including the number of replicas and any errors.

  2. Monitor the number of replicas in your Deployment:

    kubectl get deployment your_k8s_deployment -n your_k8s_namespace

    As you add messages to the RabbitMQ queue, you should see the number of replicas increase. When the queue length decreases, the number of replicas should eventually decrease as well (after the cooldown period).

  3. Important: Test the autoscaling by sending messages to your RabbitMQ queue and observing the scaling behavior of your deployment.

Disabling Kubernetes Default Autoscaler (HPA)​

If you have a Horizontal Pod Autoscaler (HPA) configured for your deployment, you need to disable it to avoid conflicts with KEDA. Add the following to your Argo CD application manifest:

ignoreDifferences:
  - group: apps
    kind: Deployment
    jsonPointers:
      - /spec/replicas
  - group: autoscaling
    kind: HorizontalPodAutoscaler
    jsonPointers:
      - /spec/maxReplicas
      - /spec/minReplicas

Troubleshooting​

  • KEDA is not scaling my deployment:
    • Check the KEDA operator logs for any errors.
    • Ensure that the RabbitMQ connection string is correct and that KEDA can connect to your RabbitMQ instance.
    • Verify that the queue name is correct.
    • Check the status of the ScaledObject using kubectl describe.
  • My deployment is scaling up and down too frequently:
    • Adjust the pollingInterval and cooldownPeriod parameters to fine-tune the scaling behavior.

Reference​