chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <ey...@yahoo-inc.com>
Subject Re: Closing .chukwa files in collector at regular fixed offset interval to avoid time interval drift
Date Tue, 31 May 2011 21:12:25 GMT
+1 for JIRA, nice to have feature.

Regards,
Eric

On 5/31/11 2:01 PM, "Ariel Rabkin" <asrabkin@gmail.com> wrote:

Also.

This sounds like something fairly site-specific. It would be really
good to have unit tests so we don't break this
feature in the future.

--Ari

On Tue, May 31, 2011 at 2:01 PM, Ariel Rabkin <asrabkin@gmail.com> wrote:
> Not quite sure I understand the use case, but if it seems useful,
> sure, open a JIRA.
>
> --Ari
>
> On Tue, May 31, 2011 at 11:33 AM, Shweta Shah <shwetapshah@gmail.com> wrote:
>> Hi,
>>
>> We have a system built on top of Chukwa that introduces the need to have all
>> sequence files for a given rotateInterval closed and available at a
>> predictable time.  Currently, we are experiencing some drift in the times
>> that the sequence files are closed due to the way the TimerTask is scheduled
>> in the SeqFileWriter class.  We would like to submit a solution that will
>> allow people to configure the time all collectors should close their files
>> for processing in a given interval, while still supporting the default
>> functionality.  We have written the code and are currently testing the
>> functionality, but would like to know if anyone has any feedback on this
>> development before submitting a JIRA ticket.
>>
>> - Shweta
>>
>
>
>
> --
> Ari Rabkin asrabkin@gmail.com
> UC Berkeley Computer Science Department
>



--
Ari Rabkin asrabkin@gmail.com
UC Berkeley Computer Science Department


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message