flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "基勇" <252637...@qq.com>
Subject flume failover only support two nodes?
Date Thu, 14 Aug 2014 03:27:34 GMT
Hello,guys
      I tested flume failover feature found failover support only two nodes, may I ask whether
it is so?‍


config file:
storm@storm01:~/apache-flume-1.5.0-bin/conf$ more flume-sink.properties 
#Name the compents on this agent
a1.sources = r1
a1.sinks = k1 k2 k3 k4
a1.channels = c1


#Describe the sinkgroups
a1.sinkgroups = g1 g2
a1.sinkgroups.g1.sinks = k1 k2 k3 k4
a1.sinkgroups.g1.processor.type = failover
a1.sinkgroups.g1.processor.priority.k1 = 10
a1.sinkgroups.g1.processor.priority.k3 = 10
a1.sinkgroups.g1.processor.priority.k4 = 10
a1.sinkgroups.g1.processor.priority.k2 = 5
a1.sinkgroups.g1.processor.maxpenalty = 10000


#a1.sinkgroups.g2.sinks = k3 k4
#a1.sinkgroups.g2.processor.type = load_balance
#a1.sinkgroups.g2.processor.backoff = true
#a1.sinkgroups.g2.processor.selector = round_robin


#Describe/config the source
a1.sources.r1.type = syslogtcp
a1.sources.r1.port = 5140
a1.sources.r1.host = localhost
a1.sources.r1.channels = c1


#Describe the sink
a1.sinks.k1.type = avro
a1.sinks.k1.channel = c1
a1.sinks.k1.hostname = 192.168.220.159
a1.sinks.k1.port = 44411


a1.sinks.k2.type = avro
a1.sinks.k2.channel = c1
a1.sinks.k2.hostname = 192.168.220.159
a1.sinks.k2.port = 44422


a1.sinks.k3.type = avro
a1.sinks.k3.channel = c1
a1.sinks.k3.hostname = 192.168.220.159
a1.sinks.k3.port = 44433


a1.sinks.k4.type = avro
a1.sinks.k4.channel = c1
a1.sinks.k4.hostname = 192.168.220.159
a1.sinks.k4.port = 44444
#Use a channel which butters events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100‍



only k3 and k2 node received ‍the data.
i stop k2 k3. k1 k4 can receive the data;


exception infomation:
2014-07-08 08:10:06,403 (SinkRunner-PollingRunner-FailoverSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)]
Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: All sinks failed to process, nothing left to failover
to
        at org.apache.flume.sink.FailoverSinkProcessor.process(FailoverSinkProcessor.java:191)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:745)
2014-07-08 08:10:11,408 (SinkRunner-PollingRunner-FailoverSinkProcessor) [INFO - org.apache.flume.sink.AbstractRpcSink.createConnection(AbstractRpcSink.java:206)]
Rpc sink k2: Building RpcClient with hostname: 192.168.220.159, port: 44422
2014-07-08 08:10:11,408 (SinkRunner-PollingRunner-FailoverSinkProcessor) [INFO - org.apache.flume.sink.AvroSink.initializeRpcClient(AvroSink.java:126)]
Attempting to create Avro Rpc client.
2014-07-08 08:10:11,408 (SinkRunner-PollingRunner-FailoverSinkProcessor) [WARN - org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:620)]
Using default maxIOWorkers
2014-07-08 08:10:11,417 (SinkRunner-PollingRunner-FailoverSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)]
Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: All sinks failed to process, nothing left to failover
to
        at org.apache.flume.sink.FailoverSinkProcessor.process(FailoverSinkProcessor.java:191)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:745)
2014-07-08 08:10:16,422 (SinkRunner-PollingRunner-FailoverSinkProcessor) [INFO - org.apache.flume.sink.AbstractRpcSink.createConnection(AbstractRpcSink.java:206)]
Rpc sink k3: Building RpcClient with hostname: 192.168.220.159, port: 44433
2014-07-08 08:10:16,423 (SinkRunner-PollingRunner-FailoverSinkProcessor) [INFO - org.apache.flume.sink.AvroSink.initializeRpcClient(AvroSink.java:126)]
Attempting to create Avro Rpc client.
2014-07-08 08:10:16,425 (SinkRunner-PollingRunner-FailoverSinkProcessor) [WARN - org.apache.flume.api.NettyAvroRpcClient.configure(NettyAvroRpcClient.java:620)]
Using default maxIOWorkers
2014-07-08 08:10:16,431 (SinkRunner-PollingRunner-FailoverSinkProcessor) [ERROR - org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:160)]
Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: All sinks failed to process, nothing left to failover
to
        at org.apache.flume.sink.FailoverSinkProcessor.process(FailoverSinkProcessor.java:191)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:745)‍
Mime
View raw message