Skip to content

Lab: Creating Topics

Using the Strimzi topic entity operator, we can define topics as CRs in Kubernetes. As with other Strimzi CRDs, the topic is referencing the cluster in which it should be created in via the label strimzi.io/cluster.

Strimzi documentation

Example: a basic topic declaration. 

apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic
metadata:
  name: topic-name-1
  labels:
    strimzi.io/cluster: my-cluster
spec:
  topicName: topic-name-1

As with other entity operators, the Strimzi topic entity operator will be monitoring the cluster for state changes and compare it to the desired state. In order to match the desired state, the topic operator will create, modify or remove topics. While it is still possible to use the Kafka Admin API to manage topics, the topic operator may revert changes which do not match its desired state. However, the topic operator will only manage topics created by itself and ignore topics created via Kafka APIs. This allows other components like MirrorMarker2 to function as if connected to a vanilla Kafka cluster while still allowing to automatically create topics.

Lab Exercise: Creating a Basic Topic

Declare a basic topic named test-0 within your personal cluster.

Solution 
apiVersion: kafka.strimzi.io/v1beta2
kind: KafkaTopic
metadata:
  name: test-0
  labels:
    strimzi.io/cluster: cluster-1
spec:
  topicName: test-0

Now create the topic by deploying the resource definition in your namespace.

Lab Exercise: Publish and Consume from the New Topic

For this exercise, we will be using the official Kafka CLI tools to create a console consumer listening on the new topic and publish a simple message to it. We will re-use the ConfigMap we created with our cluster to configure the clients. Please be aware of some Kafka CLI caveats:

  • Even though we set the bootstrap servers in the .properties file, the Kafka CLI still requires us to pass them via the parameter --boostrap-server SERVER:PORT
  • The kafka-console-producer reads messages to produce only from STDIN, requiring you to either pipe your messages to the command or write them interactively
  • We configured producer and consumers to parse the message key from each message, expecting a format key:message

Using the Kafka CLI consumer and producer, first launch a consumer for the new topic test-0 and then publish a simple text message to it.

Official Kafka CLI Tools

The official Kafka CLI tools are available in the debug image and included in ${PATH} by default excluding the .sh file name extension, e.g. the script kafka-console-consumer.sh may be invoked globally as: 

$ kafka-console-consumer

Hint 1: Launching the Consumer 
kafka-console-consumer --consumer.config /config/cluster-1/client-plaintext-noauth.properties --bootstrap-server {Bootstrap Server}:9093 --topic test-0 --group cg-0
Hint 2: Publishing a Message

Please replace MY_NAMESPACE with your personal namespace. 

echo "glados:Congratulations. Not on the test." | kafka-console-producer --producer.config /config/cluster-1/client-plaintext-noauth.properties  --bootstrap-server cluster-1-kafka-bootstrap.MY_NAMESPACE.svc:9093 --topic test-0
You can also publish messages from a sample file included in the pod’s image at /opt/samples/, e.g.: 
kafka-console-producer --producer.config /config/cluster-1/client-plaintext-noauth.properties  --bootstrap-server cluster-1-kafka-bootstrap.MY_NAMESPACE.svc:9093 --topic test-0 < /opt/samples/messages_glados_still_alive.txt