chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ariel Rabkin <asrab...@gmail.com>
Subject Re: what does Adaptor.getStreamName() do?
Date Mon, 15 Jun 2009 18:31:44 GMT
I just looked at the code.  Looks to me as though Offsets are NOT
included in the hash. The hash is based on the adaptor params, which
have already had the offset trimmed off by that point.

I can whip up a test case to nail this down for the future.

--Ari

On Mon, Jun 15, 2009 at 11:13 AM, Eric Yang<eyang@yahoo-inc.com> wrote:
> I haven't looked at the code.  md5 calculation should be calculated from
> adaptor name + stream name.  If it includes offset, then it's a bug.  Offset
> changes rapidly, and md5 signature will change if it includes offset.
>
> Regards,
> Eric
>
>
> On 6/15/09 11:04 AM, "Cheng Zhang" <chzhang@yahoo-inc.com> wrote:
>
>> In Chukwa-5 patch, the file offset is also included in md5 calculation. It
>> means that the same file could be added many times as long as the offsets are
>> different. Is it expected behavior?
>>
>> Thanks,
>> Cheng
>>
>>> -----Original Message-----
>>> From: Ariel Rabkin [mailto:asrabkin@gmail.com]
>>> Sent: Monday, June 15, 2009 10:45 AM
>>> To: chukwa-dev@hadoop.apache.org
>>> Subject: Re: what does Adaptor.getStreamName() do?
>>>
>>> As of CHUKWA-5, this is no longer true.  The agent keeps a unique name
>>> for each adaptor. (By default, the MD5 of its arguments).  So I think
>>> getStreamName() can go away.
>>>
>>> On Mon, Jun 15, 2009 at 9:51 AM, Cheng Zhang<chzhang@yahoo-inc.com> wrote:
>>>> Currently ChukwaAgent uses this method to avoid tailing the same file
>>>> more than once.
>>>>
>>>> Thanks,
>>>> Cheng
>>>>
>>>>> -----Original Message-----
>>>>> From: Ariel Rabkin [mailto:asrabkin@gmail.com]
>>>>> Sent: Sunday, June 14, 2009 9:35 PM
>>>>> To: chukwa-dev@hadoop.apache.org
>>>>> Subject: what does Adaptor.getStreamName() do?
>>>>>
>>>>> We have this method adaptor.getStreamName().  But hardly anybody calls
>>>>> it. As near as I can tell, it's only used for debugging.
>>>>>
>>>>> I would like to rename it to something like debuggingName(), or else
>>>>> remove it entirely.
>>>>> Also, AbstractAdaptor.toString() should invoke this method.
>>>>>
>>>>> --Ari
>>>>>
>>>>> --
>>>>> Ari Rabkin asrabkin@gmail.com
>>>>> UC Berkeley Computer Science Department
>>>>
>>>
>>>
>>>
>>> --
>>> Ari Rabkin asrabkin@gmail.com
>>> UC Berkeley Computer Science Department
>
>



-- 
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department

Mime
View raw message