• Description:
  This task demonstrates how to connect an EC2 instance to an Amazon MSK cluster and perform basic Kafka operations. An EC2 instance is launched within the same VPC and subnet group as the MSK brokers to ensure network connectivity. After connecting to the instance, the required Kafka client tools are installed, including Java and the Kafka binaries. Once the environment is ready, Kafka commands are used to create a topic through the provided ZooKeeper endpoint and verify it by listing all available topics. The setup is then tested by running a Kafka producer to publish messages to the topic and a Kafka consumer to read those messages in real time using the MSK broker endpoint. This confirms successful communication between the EC2 instance and the MSK cluster.

•  Step 1: Install Kafka client tools
   sudo yum install java-1.8.0 -y
   wget https://archive.apache.org/dist/kafka/3.5.1/kafka_2.13-3.5.1.tgz
   tar -xzf kafka_2.13-3.5.1.tgz
   cd kafka_2.13-3.5.1
•  Step 2: Create a topic
   bin/kafka-topics.sh --create --zookeeper <ZK-Endpoint> --replication-factor 2 --partitions 3 --topic test-topic
•  Step 3: List topics
   bin/kafka-topics.sh --list --zookeeper <ZK-Endpoint>
•  Step 4: Produce and Consume Messages
   Producer:
   bin/kafka-console-producer.sh --broker-list <MSK-Broker-Endpoint> --topic test-topic
   Type messages manually — each line is sent to Kafka.
  
   Consumer:
   bin/kafka-console-consumer.sh --bootstrap-server <MSK-Broker-Endpoint> --topic test-topic --from-beginning
   You’ll see the messages in real time

• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
• 
•