chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ariel Rabkin <asrab...@gmail.com>
Subject Re: what does Adaptor.getStreamName() do?
Date Mon, 15 Jun 2009 18:10:22 GMT
Actually, this is not an urgent fix.  The adaptor name is fixed, once
the adaptor is started.  So when the offset changes as the adaptor
runs, starting a new adaptor at the same place as the original was
started will be blocked.

This means that in normal operation, where tailing starts from 0, we
won't create spurrious duplicates if there's a tailer running.

I think this is still worth fixing, probably as CHUKWA-284, but it
isn't critical.

On Mon, Jun 15, 2009 at 11:07 AM, Ariel Rabkin<asrabkin@gmail.com> wrote:
> No.  That needs to be fixed.  I'll do that right now.
>
> On Mon, Jun 15, 2009 at 11:04 AM, Cheng Zhang<chzhang@yahoo-inc.com> wrote:
>> In Chukwa-5 patch, the file offset is also included in md5 calculation. It means
that the same file could be added many times as long as the offsets are different. Is it expected
behavior?
>>
>> Thanks,
>> Cheng
>>
>>> -----Original Message-----
>>> From: Ariel Rabkin [mailto:asrabkin@gmail.com]
>>> Sent: Monday, June 15, 2009 10:45 AM
>>> To: chukwa-dev@hadoop.apache.org
>>> Subject: Re: what does Adaptor.getStreamName() do?
>>>
>>> As of CHUKWA-5, this is no longer true.  The agent keeps a unique name
>>> for each adaptor. (By default, the MD5 of its arguments).  So I think
>>> getStreamName() can go away.
>>>
>>> On Mon, Jun 15, 2009 at 9:51 AM, Cheng Zhang<chzhang@yahoo-inc.com> wrote:
>>> > Currently ChukwaAgent uses this method to avoid tailing the same file
>>> > more than once.
>>> >
>>> > Thanks,
>>> > Cheng
>>> >
>>> >> -----Original Message-----
>>> >> From: Ariel Rabkin [mailto:asrabkin@gmail.com]
>>> >> Sent: Sunday, June 14, 2009 9:35 PM
>>> >> To: chukwa-dev@hadoop.apache.org
>>> >> Subject: what does Adaptor.getStreamName() do?
>>> >>
>>> >> We have this method adaptor.getStreamName().  But hardly anybody calls
>>> >> it. As near as I can tell, it's only used for debugging.
>>> >>
>>> >> I would like to rename it to something like debuggingName(), or else
>>> >> remove it entirely.
>>> >> Also, AbstractAdaptor.toString() should invoke this method.
>>> >>
>>> >> --Ari
>>> >>
>>> >> --
>>> >> Ari Rabkin asrabkin@gmail.com
>>> >> UC Berkeley Computer Science Department
>>> >
>>>
>>>
>>>
>>> --
>>> Ari Rabkin asrabkin@gmail.com
>>> UC Berkeley Computer Science Department
>>
>
>
>
> --
> Ari Rabkin asrabkin@gmail.com
> UC Berkeley Computer Science Department
>



-- 
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department

Mime
View raw message