首页 > 其他分享> > |NO.Z.00006|——————————|^^ 配置 ^^|——|Hadoop&kafka.V06|-------------------------------------------|kaf
|NO.Z.00006|——————————|^^ 配置 ^^|——|Hadoop&kafka.V06|-------------------------------------------|kaf
作者：互联网
[BigDataHadoop：Hadoop&kafka.V06] [BigDataHadoop.kafka][|章节一|Hadoop生态圈技术栈|kafka|kafka生产与消费|]
一、kakfa生产与消费：验证zookeeper和kafka状态
### --- 验证zookeeper状态
[root@hadoop ~]# zkServer.sh status
ZooKeeper JMX enabled by default
Using config: /opt/yanqi/servers/zookeeper/bin/../conf/zoo.cfg
Mode: standalone### --- 启动kafka
[root@hadoop ~]# kafka-server-start.sh -daemon /opt/yanqi/servers/kafka/config/server.properties### --- kafka-topics.sh 用于管理主题。
~~~ # 列出现有的主题
[root@hadoop ~]# kafka-topics.sh --list --zookeeper localhost:2181/myKafka
~~~ # 创建主题，该主题包含一个分区，该分区为Leader分区，它没有Follower分区副本。
[root@hadoop ~]# kafka-topics.sh --zookeeper localhost:2181/myKafka \
--create --topic topic_1 --partitions 1 --replication-factor 1~~~ # 查看分区信息
[root@hadoop ~]# kafka-topics.sh --zookeeper localhost:2181/myKafka --list
~~~ 输出参数
topic_1
~~~ # 查看指定主题的详细信息
[root@hadoop ~]# kafka-topics.sh --zookeeper localhost:2181/myKafka \
--describe --topic topic_1
~~~ 输出参数
Topic:topic_1 PartitionCount:1 ReplicationFactor:1 Configs:
Topic: topic_1 Partition: 0 Leader: 0 Replicas: 0 Isr: 0~~~ # 注
~~~ 当前的主题是：topic_1
~~~ 当前主题有几个分区：PartitionCount：1
~~~ 副本因子是：ReplicationFactor：1~~~ # 删除指定主题
[root@hadoop ~]# kafka-topics.sh --zookeeper localhost:2181/myKafka \
--delete --topic topic_1
~~~ 输出参数
Topic topic_1 is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.~~~ # 创建topic_2主题，有1个副本5个分区
[root@hadoop ~]# kafka-topics.sh --zookeeper localhost:2181/myKafka \
--create --topic topic_2 --partitions 5 --replication-factor 1~~~ # 查看主题
[root@hadoop ~]# kafka-topics.sh --zookeeper localhost:2181/myKafka \
--describe --topic topic_2
Topic:topic_2 PartitionCount:5 ReplicationFactor:1 Configs:
Topic: topic_2 Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: topic_2 Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: topic_2 Partition: 2 Leader: 0 Replicas: 0 Isr: 0
Topic: topic_2 Partition: 3 Leader: 0 Replicas: 0 Isr: 0
Topic: topic_2 Partition: 4 Leader: 0 Replicas: 0 Isr: 0### --- kafka-console-producer.sh用于生产消息：
~~~ # 开启生产者
[root@hadoop ~]# kafka-console-producer.sh --topic topic_1 \
--broker-list localhost:9092
~~写入参数
>yanqi——1
>yanqi——2
>yanqi——3### --- kafka-console-consumer.sh用于消费消息：
~~~ # 开启消费者
[root@hadoop ~]# kafka-console-consumer.sh --bootstrap-server localhost:9092 \
--topic topic_1
~~~ 输出参数
yanqi——1
yanqi——2
yanqi——3~~~ # 开启消费者方式二，从头消费，不按照偏移量消费
[root@hadoop ~]# kafka-console-consumer.sh --bootstrap-server localhost:9092 \
--topic topic_1 --from-beginning
~~~ 输出参数
hello world
hello world
yanqi——1
yanqi——2
yanqi——3### --- 在磁盘上查看生成消费的数据
~~~ # 该目录下打开了很多偏移量
~~~ 这些都是kafka生成消费的数据，包括主题分区详情。
[root@hadoop ~]# ls /opt/yanqi/servers/kafka/kafka-logs/
cleaner-offset-checkpoint __consumer_offsets-15 __consumer_offsets-22 __consumer_offsets-3 __consumer_offsets-37 __consumer_offsets-44 __consumer_offsets-7 topic_2-0
__consumer_offsets-0 __consumer_offsets-16 __consumer_offsets-23 __consumer_offsets-30 __consumer_offsets-38 __consumer_offsets-45 __consumer_offsets-8 topic_2-1
__consumer_offsets-1 __consumer_offsets-17 __consumer_offsets-24 __consumer_offsets-31 __consumer_offsets-39 __consumer_offsets-46 __consumer_offsets-9 topic_2-2
__consumer_offsets-10 __consumer_offsets-18 __consumer_offsets-25 __consumer_offsets-32 __consumer_offsets-4 __consumer_offsets-47 log-start-offset-checkpoint topic_2-3
__consumer_offsets-11 __consumer_offsets-19 __consumer_offsets-26 __consumer_offsets-33 __consumer_offsets-40 __consumer_offsets-48 meta.properties topic_2-4
__consumer_offsets-12 __consumer_offsets-2 __consumer_offsets-27 __consumer_offsets-34 __consumer_offsets-41 __consumer_offsets-49 recovery-point-offset-checkpoint
__consumer_offsets-13 __consumer_offsets-20 __consumer_offsets-28 __consumer_offsets-35 __consumer_offsets-42 __consumer_offsets-5 replication-offset-checkpoint
__consumer_offsets-14 __consumer_offsets-21 __consumer_offsets-29 __consumer_offsets-36 __consumer_offsets-43 __consumer_offsets-6 topic_1-0===============================END===============================
Walter Savage Landor:strove with none,for none was worth my strife.Nature I loved and, next to Nature, Art:I warm'd both hands before the fire of life.It sinks, and I am ready to depart ——W.S.Landor
来自为知笔记(Wiz)
标签：__,V06,v01,offsets,--,kafka,topic,consumer 来源： https://www.cnblogs.com/yanqivip/p/16121776.html