flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Smaine Kahlouch <smaine.kahlo...@smartjog.com>
Subject Re: Spillable memory perf tests : source stops sending
Date Tue, 24 Mar 2015 08:52:04 GMT
Hi Roshan,

By destination server i meen the flume in front of our hadoop cluster.
eg: the last component in the following flow :

source-server -> tcp/syslog -> flume-spool (memory + spooling) -> avro 
-> flume-destination (hadoop-cluster)

I didn't restart anything, i just stopped the last flume (hadoop).
Then the flume-spool should continue to spool in its memory/disks but it 
simply stopped receiving.

Regards,

-- 
Smaine Kahlouch - Engineer, Research & Engineering
Arkena | T: +33 1 5868 6196
27 Blvd Hippolyte Marquès, 94200 Ivry-sur-Seine, France
arkena.com


On 03/23/15 22:55, Roshan Naik wrote:
> Catching up on emails post vacation…
>
> - By "destination server" Do you mean the 2nd flume agent in the flow ?
> - The NullPointer is probably indicating a bug somewhere. Your logs 
> seem to indicate there a restart of the agent with the spillable 
> channel. Can you confirm if this is explicit restart or was it an 
> automatic dynamic restart due to config file update ?
>
> -roshan
>
>
> From: Smaine Kahlouch <smaine.kahlouch@smartjog.com 
> <mailto:smaine.kahlouch@smartjog.com>>
> Reply-To: "user@flume.apache.org <mailto:user@flume.apache.org>" 
> <user@flume.apache.org <mailto:user@flume.apache.org>>
> Date: Thursday, March 5, 2015 3:53 AM
> To: "user@flume.apache.org <mailto:user@flume.apache.org>" 
> <user@flume.apache.org <mailto:user@flume.apache.org>>
> Subject: Spillable memory perf tests : source stops sending
>
> Hello,
>
> I'm currently testing the spillable memory channel.
> The idea is to have a fast channel coupled with a file channel in 
> order to spool logs when our hadoop cluster is unreachable.
>
> To give you the whole architecture :
>
> source-server -> tcp/syslog -> flume-spool (memory + spooling) -> avro 
> -> flume-destination (hadoop-cluster)
>
> My test scenario is :
> - send 100 000 logs from the source server
> - stop the flume-destination during this sending
> - wait a few seconds in order to spool on the flume-spool server
> - start the flume-destination
> - check if all the logs have been received on the flume-destination 
> server.
>
> The problem is that when i stop the destination server, the source 
> server stops sending *completely*.
> Is this the normal behaviour ? Should it spool on the flume server 
> instead of stopping the source ?
>
> On my flume logs i can see the following exception :
> 05 Mar 2015 10:22:13,828 INFO 
> [SinkRunner-PollingRunner-DefaultSinkProcessor] 
> (org.apache.flume.sink.AbstractRpcSink.createConnection:217) - Rpc 
> sink sin1: Building RpcClient with hostname: 10.115.77.6, port: 55555
> 05 Mar 2015 10:22:13,828 INFO 
> [SinkRunner-PollingRunner-DefaultSinkProcessor] 
> (org.apache.flume.sink.AvroSink.initializeRpcClient:126)  - Attempting 
> to create Avro Rpc client.
> 05 Mar 2015 10:22:13,862 WARN 
> [SinkRunner-PollingRunner-DefaultSinkProcessor] 
> (org.apache.flume.api.NettyAvroRpcClient.configure:634)  - Using 
> default maxIOWorkers
> 05 Mar 2015 10:22:13,869 ERROR 
> [SinkRunner-PollingRunner-DefaultSinkProcessor] 
> (org.apache.flume.SinkRunner$PollingRunner.run:160)  - Unable to 
> deliver event. Exception follows.
> java.lang.NullPointerException
>     at 
> org.apache.flume.channel.SpillableMemoryChannel$SpillableMemoryTransaction.doRollback(SpillableMemoryChannel.java:590)
>     at 
> org.apache.flume.channel.BasicTransactionSemantics.rollback(BasicTransactionSemantics.java:168)
>     at 
> org.apache.flume.sink.AbstractRpcSink.process(AbstractRpcSink.java:394)
>     at 
> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
>     at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
>     at java.lang.Thread.run(Thread.java:745)
>
> Please find enclosed the flume configurations (spooling server and 
> destination server).
>
> Regards,
>
> -- 
> Smaine Kahlouch - Engineer, Research & Engineering
> Arkena | T: +33 1 5868 6196
> 27 Blvd Hippolyte Marquès, 94200 Ivry-sur-Seine, France
> arkena.com


Mime
View raw message