Apache Kafka 2.9.2- installation on ubuntu 14.04

posted on Nov 20th, 2016

Apache Kafka

Apache Kafka is an open-source message broker project developed by the Apache Software Foundation written in Scala. The project aims to provide a unified, high-throughput, low-latency platform for handling real-time data feeds. It is, in its essence, a "massively scalable pub/sub message queue architected as a distributed transaction log," making it highly valuable for enterprise infrastructures to process streaming data.

Pre Requirements

1) A machine with Ubuntu 14.04 LTS operating system installed.

2) Apache Kakfa 2.9.2- software (Download Here)

Apache Kafka Installation

The following steps to install Kafka on your machine.

Installation Steps

Step 1 - Installing java 7. Open a terminal (CTRL + ALT + T) and type the following sudo command.

$ sudo apt-get install openjdk-7-jdk

Check it installed properly or not.

$ java -version

Step 2 - Edit $HOME/.bashrc file by adding the java path.

$ sudo gedit $HOME/.bashrc

$HOME/.bashrc file

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64

Step 3 - Reload your changed $HOME/.bashrc settings

$ source $HOME/.bashrc

Step 4 - Creating kafka directory. Open a terminal (CTRL + ALT + T) and type the following $ sudo command.

$ sudo mkdir /usr/local/kafka

Step 5 - Change the ownership and permissions of the directory /usr/local/kafka. Here 'hduser' is an Ubuntu username.

$ sudo chown -R hduser /usr/local/kafka

$ sudo chmod -R 755 /usr/local/kafka

Step 6 - Change the directory to /home/hduser/Desktop , In my case the downloaded kafka_2.9.2- file is in /home/hduser/Desktop folder. For you it might be in /downloads folder check it.

$ cd /home/hduser/Desktop/

Step 7 - Untar the kafka_2.9.2- file.

$ tar xzf kafka_2.9.2-

Step 8 - Move the contents of kafka_2.9.2- folder to /usr/local/kafka

$ mv kafka_2.9.2-* /usr/local/kafka

Step 9 - Change the directory to /usr/local/kafka/config

$ cd /usr/local/kafka/config

Step 10 - Edit server.properties file.

$ sudo gedit server.properties

Step 11 - Add this below line to server.properties file. Save and close.

delete.topic.enable = true

Step 12 - Change the directory to /usr/local/kafka

$ cd /usr/local/kafka

Step 13 - Start the zookeeper

$ ./bin/zookeeper-server-start.sh config/zookeeper.properties

Step 14 - Open a new terminal (CTRL + ALT + T) and change the directory to /usr/local/kafka

$ cd /usr/local/kafka

Step 15 - Start the kafka server

$ ./bin/kafka-server-start.sh config/server.properties

Step 16 - Open a new terminal (CTRL + ALT + T) and just type 'jps' (without quotes). It shows all the java processes that are running.

Apache Kafka Installation on Ubuntu 14.04

Here Kafka is the kafka server and QuorumPeerMain is zookeeper process.

Step 17 - Stop zookeeper and kafka server. Just press 'CTRL + D' in the terminals of zookeeper and kafka server.

Please share this blog post and follow me for latest updates on

facebook             google+             twitter             feedburner

Previous Post                                                                                          Next Post

Labels : Kafka Single Node Single Broker Configuration   Kafka Single Node Multiple Brokers Configuration   Kafka Java API usage   Creating a Kafka topic   Modifying a Kafka topic   Deleting a Kafka topic