Apache Kafka Single Node-Single Broker

posted on Nov 20th, 2016

Apache Kafka

Apache Kafka is an open-source message broker project developed by the Apache Software Foundation written in Scala. The project aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds. It is, in its essence, a "massively scalable pub/sub message queue architected as a distributed transaction log," making it highly valuable for enterprise infrastructures to process streaming data.

Pre Requirements

1) A machine with Ubuntu 14.04 LTS operating system

2) Apache Kakfa 2.9.2-0.8.1.1 software installed. (How to install kafka on ubuntu)

Kafka Single Node Single Broker Configuration

In this configuration you have a single ZooKeeper and broker id instance. Following are the steps to configure it.

Apache Kafka Single Node Single Broker

Steps to follow

Step 1 - Open a new terminal (CTRL + ALT + T) and change the directory to /usr/local/kafka

$ cd /usr/local/kafka

Step 2 - Start the zookeeper

$ ./bin/zookeeper-server-start.sh config/zookeeper.properties

Step 3 - Open a new terminal (CTRL + ALT + T) and change the directory to /usr/local/kafka

$ cd /usr/local/kafka

Step 4 - Start the kafka server

$ ./bin/kafka-server-start.sh config/server.properties

Step 5 - Open a new terminal (CTRL + ALT + T) and just type 'jps' (without quotes). It shows all the java processes that are running.

Apache Kafka Single Node Single Broker

Here Kafka is the kafka server and QuorumPeerMain is zookeeper process.

Step 6 - Open a new terminal (CTRL + ALT + T) and change the directory to /usr/local/kafka

$ cd /usr/local/kafka

Step 7 - Create a topic.

$ ./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic Hello-Kafka

Step 8 - List a topics. (optional)

$ ./bin/kafka-topics.sh --list --zookeeper localhost:2181

Step 9 - Open a new terminal (CTRL + ALT + T) and change the directory to /usr/local/kafka

$ cd /usr/local/kafka

Step 10 - Producer

$ ./bin/kafka-console-producer.sh --broker-list localhost:9092 --topic Hello-Kafka

Step 11 - Open a new terminal (CTRL + ALT + T) and change the directory to /usr/local/kafka

$ cd /usr/local/kafka

Step 12 - Consumer

$ ./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic Hello-Kafka --from-beginning

Step 13 - Stop zookeeper, kafka server, producer and consumer. Just press 'CTRL + D' in the terminals of zookeeper, kafka server, producer and consumer.

Please share this blog post and follow me for latest updates on

facebook             google+             twitter             feedburner

Previous Post                                                                                          Next Post

Labels : Apache Kafka Installation   Kafka Single Node Multiple Brokers Configuration   Kafka Java API usage   Creating a Kafka topic   Modifying a Kafka topic   Deleting a Kafka topic