flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hari Shreedharan <hshreedha...@cloudera.com>
Subject Re: Shutdown metric question
Date Tue, 08 Oct 2013 22:20:18 GMT
No, it does not mean your channel dropped data. It just means that the sink tried to read data
695 times from the channel, but the channel could only return 471 - obviously because only
that many events were written in the first place. 


Thanks,
Hari


On Tuesday, October 8, 2013 at 12:03 PM, DSuiter RDX wrote:

> Hi all,
> 
> We have a TCP source channeled to an Avro sink, which is connecting to an Avro source
channeled to HDFS sink. Everything is flowing, but I noticed this when it shut down: 
> 
> Shutdown Metric for type: CHANNEL, name: memory_RT_Tier2_Channel. channel.event.put.attempt
== 471
> Shutdown Metric for type: CHANNEL, name: memory_RT_Tier2_Channel. channel.event.put.success
== 471
> Shutdown Metric for type: CHANNEL, name: memory_RT_Tier2_Channel. channel.event.take.attempt
== 695
> Shutdown Metric for type: CHANNEL, name: memory_RT_Tier2_Channel. channel.event.take.success
== 471
> 
> Does the difference between the channel.take.attempt == 695 and take.success == 471 indicate
that my channel dropped data, or does it indicate that the channel tried to grab from the
source but did not get data from it? There is a period when the Avro RPC call connects where
it is not fully socketed, so maybe the channel is grabbing air? If so, that is ok. If it indicates
data got lost, that is not acceptable and hopefully someone can help me correct the root cause
there. 
> 
> Thanks!
> Devin Suiter
> Jr. Data Solutions Software Engineer
> 
> 100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212
> Google Voice: 412-256-8556 | www.rdx.com (http://www.rdx.com/) 


Mime
View raw message