Lab: Install Separate Controller Cluster

Task 1: Uninstall Cluster

Delete the first Cluster in your namespace.

Hint 1: How To

Remove your Kafka and the KafkaNodePool CR from the K8s cluster. Remove all PVCs assigned to your cluster.

Hint 2: Command

Execute a delete command on the cluster CR.

Hint 3: Full Command

kubectl delete -n {your namespace} cluster.yml

Task 2: Create the CR

Create a combined cluster-kraft.yml file in your workspace. It should:

• Create a Kafka CR using the base CR from the last exercise including monitoring config.
• Change the port of the plain listener to 9092
• Create a kafkaNodePool CR with the controller role. It should have:
  • controller role
  • 3 replicas
  • 50Gi JBOD storage with shared kraft Metadata
  • 2Gi Memory request and limit
  • 200m CPU request
• Create a kafkaNodePool CR with the broker role. It should have:
  • broker role
  • 3 replicas
  • 100Gi JBOD storage with shared kraft Metadata
  • 4Gi Memory request and limit
  • 500m CPU request

Use the official Strimzi documentation to perform this task.

https://strimzi.io/docs/operators/latest/deploying

More specific:

https://strimzi.io/docs/operators/latest/deploying#assembly-kraft-mode-str

And you may use:

https://strimzi.io/docs/operators/latest/deploying#config-examples-str

Hint 1

If you need a good starting point look into the examples directory of the strimzi repo. It contains example KAFKA configs. For our usecase we also work with persistance using JBOD

Hint 2

https://github.com/strimzi/strimzi-kafka-operator/blob/0.48.0/examples/kafka/kafka-jbod.yaml

Hint 3

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
  name: controller
  labels:
    strimzi.io/cluster: cluster-1
spec:
  replicas: 3
  roles:
    - controller
  storage:
    type: jbod
    volumes:
      - id: 0
        type: persistent-claim
        size: 50Gi
        kraftMetadata: shared
        deleteClaim: false
        class: managed-csi
  resources:
    requests:
      memory: 2Gi
      cpu: "200m"
    limits:
      memory: 2Gi
---
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaNodePool
metadata:
  name: broker
  labels:
    strimzi.io/cluster: cluster-1
spec:
  replicas: 3
  roles:
    - broker
  storage:
    type: jbod
    volumes:
      - id: 0
        type: persistent-claim
        size: 100Gi
        kraftMetadata: shared
        deleteClaim: false
        class: managed-csi
  resources:
    requests:
      memory: 4Gi
      cpu: "500m"
    limits:
      memory: 4Gi
---
apiVersion: kafka.strimzi.io/v1beta2
kind: Kafka
metadata:
  name: cluster-1
  annotations:
    strimzi.io/node-pools: enabled
    strimzi.io/kraft: enabled
spec:
  kafkaExporter:
    topicRegex: "^[^_].*$"
    groupRegex: "^[^_].*$"
  kafka:
    version: 4.1.0
    listeners:
      - name: plain
        port: 9092
        type: internal
        tls: false
    config:
      offsets.topic.replication.factor: 3
      transaction.state.log.replication.factor: 3
      transaction.state.log.min.isr: 2
      default.replication.factor: 3
      min.insync.replicas: 2
    metricsConfig:
      type: jmxPrometheusExporter
      valueFrom:
        configMapKeyRef:
          name: kafka-metrics
          key: kafka-metrics-config.yml
  entityOperator:
    topicOperator: {}
    userOperator: {}

Task 3: Deploy the Cluster

Apply the and view the Cluster in k9s and Lenses.io.

Hint 1

Apply the cluster-kraft.yml file. view the pods with k9s. Visit the lenses UI.

Hint 2

1. kubectl apply -n {your namespace} cluster-kraft.yml
2. k9s
3. Visit {link}

Hint 3

aks the trainer if something went wrong.