flume-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From DSuiter RDX <dsui...@rdx.com>
Subject Re: Shutdown metric question
Date Wed, 09 Oct 2013 12:40:25 GMT
Thanks Hari, that was what I was hoping to hear. Based on the first two
lines, that was what I had expected, but wanted to confirm before trusting
anything of value to the agent recipe as it stands.

Sincerely,
*Devin Suiter*
Jr. Data Solutions Software Engineer
100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212
Google Voice: 412-256-8556 | www.rdx.com


On Tue, Oct 8, 2013 at 6:20 PM, Hari Shreedharan
<hshreedharan@cloudera.com>wrote:

>  No, it does not mean your channel dropped data. It just means that the
> sink tried to read data 695 times from the channel, but the channel could
> only return 471 - obviously because only that many events were written in
> the first place.
>
>
> Thanks,
> Hari
>
> On Tuesday, October 8, 2013 at 12:03 PM, DSuiter RDX wrote:
>
> Hi all,
>
> We have a TCP source channeled to an Avro sink, which is connecting to an
> Avro source channeled to HDFS sink. Everything is flowing, but I noticed
> this when it shut down:
>
> Shutdown Metric for type: CHANNEL, name: memory_RT_Tier2_Channel.
> channel.event.put.attempt == 471
> Shutdown Metric for type: CHANNEL, name: memory_RT_Tier2_Channel.
> channel.event.put.success == 471
> Shutdown Metric for type: CHANNEL, name: memory_RT_Tier2_Channel.
> channel.event.take.attempt == 695
> Shutdown Metric for type: CHANNEL, name: memory_RT_Tier2_Channel.
> channel.event.take.success == 471
>
> Does the difference between the channel.take.attempt == 695 and
> take.success == 471 indicate that my channel dropped data, or does it
> indicate that the channel tried to grab from the source but did not get
> data from it? There is a period when the Avro RPC call connects where it is
> not fully socketed, so maybe the channel is grabbing air? If so, that is
> ok. If it indicates data got lost, that is not acceptable and hopefully
> someone can help me correct the root cause there.
>
> Thanks!
> *Devin Suiter*
> Jr. Data Solutions Software Engineer
> 100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212
> Google Voice: 412-256-8556 | www.rdx.com
>
>
>

Mime
View raw message