Flume代理无法传送事件

Question

我有3个代理。1个运行在使用内存通道的windows上，另外2个运行在Linux上，使用文件通道从windows代理获取数据并放入Hbase。

谁能建议一下为什么会发生下面的错误，以及采取什么步骤来阻止它？

2013-12-23 14:50:15,290 (SinkRunner-PollingRunner-DefaultSinkProcessor) [DEBUG - org.apache.flume.sink.AvroSink.destroyConnection(AvroSink.java:199)] Avro sinksink1 closing avro client: NettyAvroRpcClient { host: 192.168.101.232, port: 3001 }
2013-12-23 14:50:15,290 (SinkRunner-PollingRunner-DefaultSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)] Unable to deliver event.
Exception follows.org.apache.flume.EventDeliveryException: Failed to send events
        at org.apache.flume.sink.AvroSink.process(AvroSink.java:325)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Unknown Source)
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: 192.168.101.232, port: 3001 }: Failed to send batch
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:236)
        at org.apache.flume.sink.AvroSink.process(AvroSink.java:309)        ... 3 more
Caused by: org.apache.flume.EventDeliveryException: NettyAvroRpcClient { host: 192.168.101.232, port: 3001 }: Handshake timed out after 20000ms
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:280)
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:224)    ... 4 more
Caused by: java.util.concurrent.TimeoutException
        at java.util.concurrent.FutureTask.get(Unknown Source)
        at org.apache.flume.api.NettyAvroRpcClient.appendBatch(NettyAvroRpcClient.java:278)

以下是我的windows配置文件

a1.sources = source1
a1.channels = channel1 channel2
a1.sinks = sink1 sink2

a1.sources.source1.handler = com.flume.handler.DynamicJSONHandler
a1.sources.source1.type = org.apache.flume.source.http.HTTPSource
a1.sources.source1.bind = 192.168.101.29
a1.sources.source1.port = 2001

a1.channels.channel1.type = org.apache.flume.channel.MemoryChannel
a1.channels.channel1.capacity = 1000
a1.channels.channel1.transactionCapacity = 1000

a1.sinks.sink1.type = org.apache.flume.sink.AvroSink
a1.sinks.sink1.hostname = 192.168.101.232
a1.sinks.sink1.port = 3001

a1.channels.channel2.type = org.apache.flume.channel.MemoryChannel
a1.channels.channel2.capacity = 1000
a1.channels.channel2.transactionCapacity = 1000

a1.sinks.sink2.type = org.apache.flume.sink.AvroSink
a1.sinks.sink2.hostname = 192.168.101.233
a1.sinks.sink2.port = 3001

a1.sources.source1.channels = channel1 channel2
a1.sinks.sink1.channel = channel1
a1.sinks.sink2.channel = channel2

Answer 1

AvroSink旨在将数据写入另一个flume代理，该代理具有侦听该特定源的Avro source。Avro接收器启动与它所连接的源的握手。但是看起来握手在20秒后就超时了--所以你的网络很可能面临着严重的延迟问题。您是否在运行源的代理上看到任何问题？