1、部署
本节介绍生产就绪的部署解决方案。一般来说,我们需要部署一个没有单点故障的弹性RocketMQ集群。
Production-ready 部署
Name Server
要确保在一个实例崩溃时群集仍可运行,建议使用两个或更多Name Server实例。只要有一个Name Server实例处于活动状态,整个群集就会保持服务状态。
Name Server遵循无共享设计范例。Brokers 将心跳数据发送到所有Name Server。producer和consumer可以在发送/消费消息时从任何可用的Name Server查询元数据。
Broker
Broker可以根据他们的角色分为两类:master和slave。主代理提供RW访问,而从代理只接受读访问。
要部署没有单点故障的高可用性RocketMQ集群,应部署一系列Broker集。代理集包含一个master,其brokerMd设置为0,多个slave具有非零brokerID。一组中的所有代理都具有相同的brokerName。在严重的情况下,我们应该在一个Broker集合中至少有两个Broker。每个主题都驻留在两个或更多Broker中。
组态
部署RocketMQ集群时,建议的配置如下所示:
经纪人配置
物业名称 | 默认值 | 细节 |
---|---|---|
的listenPort | 10911 | 为客户端监听端口 |
namesrvAddr | 空值 | 名称服务器地址 |
brokerIP1 | 网络接口的InetAddress | 如果有多个地址,应配置 |
brokerName | 空值 | 经纪人姓名 |
brokerClusterName | DefaultCluster | 这个经纪人属于哪个集群 |
brokerId | 0 | 经纪人ID,0表示主人,正整数表示奴隶 |
storePathCommitLog | $ HOME /存储/ commitlog / | 提交日志的文件路径 |
storePathConsumerQueue | $ HOME /存储/ consumequeue / | 消耗队列的文件路径 |
mapedFileSizeCommitLog | 1024 * 1024 * 1024(1G) | 提交日志的映射文件大小 |
deleteWhen | 04 | 何时删除超出保留时间的提交日志 |
fileReserverdTime | 72 | 在删除之前保留commitlog的小时数 |
brokerRole | ASYNC_MASTER | SYNC_MASTER / ASYNC_MASTER / SLVAE |
flushDiskType | ASYNC_FLUSH | {SYNC_FLUSH / ASYNC_FLUSH}。在确认生产者之前,SYNC_FLUSH模式的代理将每个消息刷新到磁盘上。另一方面,ASYNC_FLUSH模式的代理利用了组提交,实现了更好的性能。 |
CLI管理工具
RocketMQ提供CLI(命令行界面)管理工具带来查询,管理和诊断各种问题。
如何使用
管理工具非常易于使用。这里,为了演示目的,假设linux环境。
将目录更改为$ {PACKAGE} / bin,命令bash mqadmin,您应该看到以下帮助菜单:
The most commonly used mqadmin commands are:
updateTopic Update or create topic
deleteTopic Delete topic from broker and NameServer
updateSubGroup Update or create subscription group
deleteSubGroup Delete subscription group from broker
updateBrokerConfig Update broker's config
updateTopicPerm Update topic perm
topicRoute Examine topic route info
topicStatus Examine topic Status info
topicClusterList get cluster info for topic
brokerStatus Fetch broker runtime status data
queryMsgById Query Message by Id
queryMsgByKey Query Message by Key
queryMsgByUniqueKey Query Message by Unique key
queryMsgByOffset Query Message by offset
queryMsgByUniqueKey Query Message by Unique key
printMsg Print Message Detail
sendMsgStatus Send msg to broker
brokerConsumeStats Fetch broker consume stats data
producerConnection Query producer's socket connection and client version
consumerConnection Query consumer's socket connection, client version and subscription
consumerProgress Query consumers's progress, speed
consumerStatus Query consumer's internal data structure
cloneGroupOffset Clone offset from other group
clusterList List all of clusters
topicList Fetch all topic list from name server
updateKvConfig Create or update KV config
deleteKvConfig Delete KV config
wipeWritePerm Wipe write perm of broker in all name server
resetOffsetByTime Reset consumer offset by timestamp(without client restart)
updateOrderConf Create or update or delete order conf
cleanExpiredCQ Clean expired ConsumeQueue on broker.
cleanUnusedTopic Clean unused topic on broker
startMonitoring Start Monitoring
statsAll Topic and Consumer tps stats
syncDocs Synchronize wiki and issue to github.com
allocateMQ Allocate MQ
checkMsgSendRT Check message send response time
clusterRT List All clusters Message Send RT
请参阅'mqadmin帮助 '有关特定命令的更多信息。如果您想获得有关像'clusterList'这样的特定命令的更多信息,只需键入bash mqadmin help clusterList,您将看到:
usage: mqadmin clusterList [-h] [-i <arg>] [-m] [-n <arg>]
-h,--help Print help
-i,--interval <arg> specify intervals numbers, it is in seconds
-m,--moreStats Print more stats
-n,--namesrvAddr <arg> Name server address list, eg: 192.168.0.1:9876;192.168.0.2:9876
Replication mode复制模式
为了确保不会丢失成功发布的消息,RocketMQ提供了一种复制模式,通过两种复制方式获得更强的持久性和更高的可用性:同步和异步。
复制:同步/异步Broker
与许多复制系统一样,同步Broker会等待,直到确认之前将提交日志复制到从属。相反,异步代理在主服务器上处理消息后立即返回。
如何配置
在conf文件夹下有RocketMQ发行版附带的三个预构建配置供您参考:
2m-2s-sync
2m-2s-async
2m-noslave
注意:所有配置都使用ASYNC_FLUSH。
部署
以2m-2s-sync的部署为例。首先,启动两个name server,如“快速入门”部分所示。假设它们的IP是192.168.0.2和192.168.0.3。
然后启动经纪人(假设二进制RocketMQ在/ home / rocketmq / dist)
>cd /home/rocketmq/dist/bin
>bash mqbroker -c ../conf/2m-2s-sync/broker-a.properties -n 192.168.0.2:9876,192.168.0.3:9876
>bash mqbroker -c ../conf/2m-2s-sync/broker-a-s.properties -n 192.168.0.2:9876,192.168.0.3:9876
>bash mqbroker -c ../conf/2m-2s-sync/broker-b.properties -n 192.168.0.2:9876,192.168.0.3:9876
>bash mqbroker -c ../conf/2m-2s-sync/broker-b-s.properties -n 192.168.0.2:9876,192.168.0.3:9876
How to verify
Execute the following command to verify according to the CLI section:
> bash mqadmin clusterlist
2、CLI管理工具
RocketMQ提供CLI管理工具带来查询,管理和诊断各种问题。
如何获得它
管理工具随RocketMQ一起发货。无论您是自己下载预构建的二进制版本还是从源代码构建,都可以使用该工具包。
如果您想查看源代码,请参阅rocketmq-tools模块
如何使用
管理工具非常用户友好。这里,为了演示目的,假设* nix环境。
将目录更改为$ {PACKAGE} / bin,命令bash mqadmin
,您应该看到以下帮助菜单。
The most commonly used mqadmin commands are:
updateTopic Update or create topic
deleteTopic Delete topic from broker and NameServer.
updateSubGroup Update or create subscription group
deleteSubGroup Delete subscription group from broker.
updateBrokerConfig Update broker's config
updateTopicPerm Update topic perm
topicRoute Examine topic route info
topicStatus Examine topic Status info
topicClusterList Get cluster info for topic
brokerStatus Fetch broker runtime status data
queryMsgById Query Message by Id
queryMsgByKey Query Message by Key
queryMsgByUniqueKey Query Message by Unique key
queryMsgByOffset Query Message by offset
queryMsgByUniqueKey Query Message by Unique key
printMsg Print Message Detail
sendMsgStatus Send msg to broker.
brokerConsumeStats Fetch broker consume stats data
producerConnection Query producer's socket connection and client version
consumerConnection Query consumer's socket connection, client version and subscription
consumerProgress Query consumers's progress, speed
consumerStatus Query consumer's internal data structure
cloneGroupOffset Clone offset from other group.
clusterList List all of clusters
topicList Fetch all topic list from name server
updateKvConfig Create or update KV config.
deleteKvConfig Delete KV config.
wipeWritePerm Wipe write perm of broker in all name server
resetOffsetByTime Reset consumer offset by timestamp(without client restart).
updateOrderConf Create or update or delete order conf
cleanExpiredCQ Clean expired ConsumeQueue on broker.
cleanUnusedTopic Clean unused topic on broker.
startMonitoring Start Monitoring
statsAll Topic and Consumer tps stats
syncDocs Synchronize wiki and issue to github.com
allocateMQ Allocate MQ
checkMsgSendRT Check message send response time
clusterRT List All clusters Message Send RT
See 'mqadmin help <command>' for more information on a specific command.
如您所见,列出了最常用的命令,并附有简要说明。要获得每个命令的详细手册,bash mqadmin help <command>
。例如,命令bash mqadmin help clusterList
将显示以下帮助文本:
usage: mqadmin clusterList [-h] [-i <arg>] [-m] [-n <arg>]
-h,--help Print help
-i,--interval <arg> specify intervals numbers, it is in seconds
-m,--moreStats Print more stats
-n,--namesrvAddr <arg> Name server address list, eg: 192.168.0.1:9876;192.168.0.2:9876
帮助文本列出了每个选项的可能选项和解释。