分布式消息队列kafka系列介绍 — 安装运行

• 作者 inter12
• 4 五月, 2014
• 暂无评论

第一步 包下载

地址 ： https://www.apache.org/dyn/closer.cgi?path=/kafka/0.8.1.1/kafka_2.9.2-0.8.1.1.tgz
tar -zxvf kafka_2.9.2-0.8.1.1.tgz
cd kafka_2.9.2-0.8.1.1

第二步 启动服务

启动zookeeper ：
bin/zookeeper-server-start.sh config/zookeeper.properties
…
[2014-05-04 13:56:57,153] INFO binding to port 0.0.0.0/0.0.0.0:2181 (org.apache.zookeeper.server.NIOServerCnxn)
[2014-05-04 13:56:57,171] INFO Reading snapshot /tmp/zookeeper/version-2/snapshot.0 (org.apache.zookeeper.server.persistence.FileSnap)
[2014-05-04 13:56:57,200] INFO Snapshotting: 67 (org.apache.zookeeper.server.persistence.FileTxnSnapLog)

启动Kafka：
bin/kafka-server-start.sh config/server.properties
…
[2014-05-04 13:57:20,520] INFO 0 successfully elected as leader (kafka.server.ZookeeperLeaderElector)
[2014-05-04 13:57:20,735] INFO Registered broker 0 at path /brokers/ids/0 with address localhost:9092. (kafka.utils.ZkUtils$)
[2014-05-04 13:57:20,736] INFO New leader is 0 (kafka.server.ZookeeperLeaderElector$LeaderChangeListener)
[2014-05-04 13:57:20,754] INFO [Kafka Server 0], started (kafka.server.KafkaServer)

第三步 创建一个topic：

zhaoming@zhaoming:~/install/soft/kafaka/kafka_2.9.2-0.8.1.1$ bin/kafka-topics.sh –create –zookeeper localhost:2181 –replication-factor 1 –partitions 1 –topic test_topic
Created topic “test_topic”.

查看是否创建成功：
zhaoming@zhaoming:~/install/soft/kafaka/kafka_2.9.2-0.8.1.1$ bin/kafka-topics.sh –list –zookeeper localhost:2181
test_topic

第四步 发送一个消息

zhaoming@zhaoming:~/install/soft/kafaka/kafka_2.9.2-0.8.1.1$ bin/kafka-console-producer.sh –broker-list localhost:9092 –topic test_topic
SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder”.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
this is first message !

第五步 建立一个消费者

> zhaoming@zhaoming:~/install/soft/kafaka/kafka_2.9.2-0.8.1.1$ bin/kafka-console-consumer.sh –zookeeper localhost:2181 –topic test_topic –from-beginning
SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder”.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
this is first message !

第六步 建立一个多broker集群

> cp config/server.properties config/server-1.properties
> cp config/server.properties config/server-2.properties

修改如下配置信息
config/server-1.properties:
broker.id=1
port=9093
log.dir=/tmp/kafka-logs-1

config/server-2.properties:
broker.id=2
port=9094
log.dir=/tmp/kafka-logs-2

broker.id 是唯一标示，因为在单机运行集群，所以修改下端口地址和日志存放目录

启动这两个broker
> bin/kafka-server-start.sh config/server-1.properties &
…
> bin/kafka-server-start.sh config/server-2.properties &
…

建立一个集群的topic
zhaoming@zhaoming:~/install/soft/kafaka/kafka_2.9.2-0.8.1.1$ bin/kafka-topics.sh –create –zookeeper localhost:2181 –replication-factor 3 –partitions 1 –topic replicated-topic
Created topic “replicated-topic”.

查看集群情况
zhaoming@zhaoming:~/install/soft/kafaka/kafka_2.9.2-0.8.1.1$ bin/kafka-topics.sh –describe –zookeeper localhost:2181 –topic replicated-topic
Topic:replicated-topic PartitionCount:1 ReplicationFactor:3 Configs:
Topic: replicated-topic Partition: 0 Leader: 1 Replicas: 1,0,2 Isr: 1,0,2

第一行信息：一个分区，复制因子是3
第二行信息：名字是replicated-topic 分区0 ，多个broker的leader是boker.id为1 的那个broker ， Replicas：复制且被leader管理的broker ，Isr：目前存活且被leader管理的broker

查看下没有做复制的topic
zhaoming@zhaoming:~/install/soft/kafaka/kafka_2.9.2-0.8.1.1$ bin/kafka-topics.sh –describe –zookeeper localhost:2181 –topic test_topic
Topic:test_topic PartitionCount:1 ReplicationFactor:1 Configs:
Topic: test_topic Partition: 0 Leader: 0 Replicas: 0 Isr: 0

一个分区，复制因子是1 leader是boker.id为1的broker，复制且存活的broker都是0

发送集群消息
zhaoming@zhaoming:~/install/soft/kafaka/kafka_2.9.2-0.8.1.1$ bin/kafka-console-producer.sh –broker-list localhost:9092 –topic replicated-topic
SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder”.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
this is replicated messafe

接受集群消息
zhaoming@zhaoming:~/install/soft/kafaka/kafka_2.9.2-0.8.1.1$ bin/kafka-console-consumer.sh –zookeeper localhost:2181 –from-beginning –topic replicated-topic
SLF4J: Failed to load class “org.slf4j.impl.StaticLoggerBinder”.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
this is replicated messafe

Tags: linux