Lab: Configure Cruise Control

Task 1: Add Cruise Control to Kafka

Add Cruise Control to your Kafka CR.

It should limit to 1Gi of RAM and 200m of CPU.

No additional goals should be set.

Use the kafka-cluster-cluster-ca-cert and the cluster-tls-managed.

Enhance the criuse control config with:

        autoRebalance:
          - mode: add-brokers
            template:
              name: add-brokers-rebalancing-template
          - mode: remove-brokers
            template:
              name: remove-brokers-rebalancing-template

Create the Rebalance CRs:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaRebalance
metadata:
  name: add-brokers-rebalancing-template
  annotations:
    strimzi.io/rebalance-template: "true"
# no goals specified, using the default goals from the Cruise Control configuration
spec: {}
---
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaRebalance
metadata:
  name: remove-brokers-rebalancing-template
  annotations:
    strimzi.io/rebalance-template: "true"
# no goals specified, using the default goals from the Cruise Control configuration
spec: {}

https://strimzi.io/blog/2020/06/15/cruise-control/

Hint 1

Check the Deploying Cruise Control Chapter of the first link

Hint 2

apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
...
spec:
  ...
  kafka:
  ...
  entityOperator:
    ...
  cruiseControl:
    resources:
      requests:
        memory: 1Gi
        cpu: "200m"
      limits:
        memory: 1Giw
    autoRebalance:
      - mode: add-brokers
        template:
          name: add-brokers-rebalancing-template
      - mode: remove-brokers
        template:
          name: remove-brokers-rebalancing-template

Hint 3

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
  name: controller
  labels:
    strimzi.io/cluster: cluster-1
spec:
  replicas: 3
  roles:
    - controller
  storage:
    type: jbod
    volumes:
      - id: 0
        type: persistent-claim
        size: 50Gi
        kraftMetadata: shared
        deleteClaim: false
        class: managed-csi
  resources:
    requests:
      memory: 2Gi
      cpu: "200m"
    limits:
      memory: 2Gi
---
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
  name: broker
  labels:
    strimzi.io/cluster: cluster-1
spec:
  replicas: 3
  roles:
    - broker
  storage:
    type: jbod
    volumes:
      - id: 0
        type: persistent-claim
        size: 100Gi
        kraftMetadata: shared
        deleteClaim: false
        class: managed-csi
  resources:
    requests:
      memory: 4Gi
      cpu: "500m"
    limits:
      memory: 4Gi
---
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: cluster-1
  annotations:
    strimzi.io/node-pools: enabled
    strimzi.io/kraft: enabled
spec:
  kafkaExporter:
      topicRegex: "^[^_].*$"
      groupRegex: "^[^_].*$"
  kafka:
    version: 4.1.0
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
        authentication:
          type: scram-sha-512
      - name: tls
        port: 9093
        type: internal
        tls: true
        authentication:
          type: scram-sha-512
        configuration:
          brokerCertChainAndKey:
            secretName: cluster-1-tls-managed
            certificate: tls.crt
            key: tls.key
    authorization:
      type: simple
      superUsers:
        - client-01
    config:
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      default.replication.factor: 3
      min.insync.replicas: 2
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          name: kafka-metrics
          key: kafka-metrics-config.yml
  entityOperator:
    topicOperator: {}
    userOperator: {}
  cruiseControl:
    resources:
      requests:
        memory: 1Gi
        cpu: "200m"
      limits:
        memory: 1Giw
    autoRebalance:
      - mode: add-brokers
        template:
          name: add-brokers-rebalancing-template
      - mode: remove-brokers
        template:
          name: remove-brokers-rebalancing-template

Task 2: Perform a Rebalance

Add the rebalance resource:

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaRebalance
metadata:
  name: my-rebalance
  labels:
    strimzi.io/cluster: kafka-cluster
# no goals specified, using the default goals from the Cruise Control configuration
spec: {}

Follow the blog post: https://strimzi.io/blog/2020/06/15/cruise-control/

kubectl describe kafkarebalance my-rebalance -n {your namepace} 
kubectl annotate kafkarebalance my-rebalance strimzi.io/rebalance=approve -n {your namepace}
kubectl describe kafkarebalance my-rebalance -n {your namepace}