incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <ey...@yahoo-inc.com>
Subject Re: rotate done
Date Wed, 24 Feb 2010 03:02:52 GMT
This is because Chukwa doesn't use HDFS append feature.  Any data that
arrived late, they will be labeled as .#.evt.  In this case, it looks like
there were two spills of data after daily rolling is done.  It would be nice
to have a scheduler to repack the data again after spill, but this feature
has not been implemented.

Regards,
Eric

On 2/23/10 2:20 PM, "Corbin Hoenes" <corbin@tynt.com> wrote:

> We are seeing files like this after the rotation jobs have run:
> 
> MyFile_20100222.1.evt  (150 MB)
> MyFile_20100222.2.evt  (5 MB)
> MyFile_DailyDone_20100222.1.evt  (64 GB)
> 
> What are the smaller files for?
> 


Mime
View raw message