incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Corbin Hoenes <cor...@tynt.com>
Subject Re: rotate done
Date Wed, 24 Feb 2010 03:33:18 GMT
Thanks Eric!

Sent from my iPhone

On Feb 23, 2010, at 8:02 PM, Eric Yang <eyang@yahoo-inc.com> wrote:

> This is because Chukwa doesn't use HDFS append feature.  Any data that
> arrived late, they will be labeled as .#.evt.  In this case, it  
> looks like
> there were two spills of data after daily rolling is done.  It would  
> be nice
> to have a scheduler to repack the data again after spill, but this  
> feature
> has not been implemented.
>
> Regards,
> Eric
>
> On 2/23/10 2:20 PM, "Corbin Hoenes" <corbin@tynt.com> wrote:
>
>> We are seeing files like this after the rotation jobs have run:
>>
>> MyFile_20100222.1.evt  (150 MB)
>> MyFile_20100222.2.evt  (5 MB)
>> MyFile_DailyDone_20100222.1.evt  (64 GB)
>>
>> What are the smaller files for?
>>
>

Mime
View raw message