今天接到客户是咨询，要对现有的数据进行分流，虽然感觉flume实现没问题，但是心里总不踏实，要是出什么幺蛾子就打脸了，随后做了一个小测试，测试的结果也验证了我的想法，废话不多说，下面直接上配置：

#配置文件名为：flume-test.conf
# list the sources, sinks and channels in the agent
a1.sources= source_1
a1.sinks= fs_sink_local_file_1 fs_sink_local_file_2
a1.channels= fs_channel_memory_1 fs_channel_memory_2

# source from  spooldir
a1.sources.source_1.type = spooldir
a1.sources.source_1.spoolDir = /home/users/flume/data
a1.sources.source_1.fileHeader = true


#channel for source configure
a1.sources.source_1.channels= fs_channel_memory_1 fs_channel_memory_2

##sinks
#sink1
a1.sinks.fs_sink_local_file_1.channel = fs_channel_memory_1
a1.sinks.fs_sink_local_file_1.type = file_roll
a1.sinks.fs_sink_local_file_1.sink.directory = /home/users/flume/dataA
#sink2
a1.sinks.fs_sink_local_file_2.channel = fs_channel_memory_2
a1.sinks.fs_sink_local_file_2.type = file_roll
a1.sinks.fs_sink_local_file_2.sink.directory = /home/users/flume/dataB

###channels
#channels1
a1.channels.fs_channel_memory_1.type = memory
a1.channels.fs_channel_memory_1.capacity = 5000000
a1.channels.fs_channel_memory_1.transactionCapacity = 5000000
a1.channels.fs_channel_memory_1.byteCapacityBufferPercentage = 40
a1.channels.fs_channel_memory_1.byteCapacity = 8000000000
a1.channels.fs_channel_memory_1.keep-alive = 30

#channels2
a1.channels.fs_channel_memory_2.type = memory
a1.channels.fs_channel_memory_2.capacity = 5000000
a1.channels.fs_channel_memory_2.transactionCapacity = 5000000
a1.channels.fs_channel_memory_2.byteCapacityBufferPercentage = 40
a1.channels.fs_channel_memory_2.byteCapacity = 8000000000
a1.channels.fs_channel_memory_2.keep-alive = 30

可以直接拿去贴就可以跑的配置，启动命令如下：

flume-ng agent -n a1 -f /path/to/conf/flume-test.conf -Xmx100M

以上flume分流复制实现的功能为：把 /home/users/flume/data目录下的文件分别复制到/home/users/flume/dataA和/home/users/flume/dataB，既然这个功能能这样实现，那么扩展到其他source，sink和channel都是没问题的，
ok，今天就到这，
小小菜鸟，不喜勿喷