Running Kafka Producers and Consumers in Secured Environment

Apache Kafka allows to run in secured mode using Kerberos Authentication.
To run Kafka producers and consumers, we need follow to the following steps:

Step 1. created a properties file say kafka.properties to add the following lines:

security.protocol=SASL_PLAINTEXT
sasl.kerberos.service.name=kafka

Step 2. Created a file say kafka.jaas to add the authentication details:

KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useTicketCache=true;
};

Step 3. Added kafka.jaas file to the Kafka environment i.e., using KAFKA_OPTS environment variable in shell (both producer and consumer):

export KAFKA_OPTS="-Djava.security.auth.login.config=/path/to/kafka.jaas.conf"

Step 4: Started a kafka-console-producer:

kafka-console-producer --brokers0list <brokers list> --topic <topic Name> --producer.config kafka.properties

Step 5: Started a kafka-console-consumer:

kafka-console-consumer --bootstrap-server <brokers list> --topic <topic Name> --consumer.config kafka.properties --new-consumer

Step 6: Type some sample messages in the Producer console and check the messages are available in Consumer console.

Note: provide brokers list as brokerhost:port,brokerhost:port,...