chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jerome Boulon <jbou...@netflix.com>
Subject Re: Data loss on collector side
Date Tue, 02 Nov 2010 16:53:08 GMT
Hi,
The HDFS writer is not doing that but the LocalWriter (HDFSMover) is exactly doing this.
/Jerome.


On 11/2/10 9:48 AM, "Jaydeep Ayachit" <jaydeep_ayachit@persistent.co.in> wrote:

Hello,

When collector is in middle of committing chunks to sequence file, if hdfs becomes unavailable,
the collector bails out. What happens to .chukwa file the collector is working on? This file
will not be renamed to .done file. The file as such would become orphan as next time collector
starts, it will not carry on with this file.

Is there any process, that looks for orphan files and takes action on them? (like rename to
.done)

Thanks,
Jaydeep


DISCLAIMER ========== This e-mail may contain privileged and confidential information which
is the property of Persistent Systems Ltd. It is intended only for the use of the individual
or entity to which it is addressed. If you are not the intended recipient, you are not authorized
to read, retain, copy, print, distribute or use this message. If you have received this communication
in error, please notify the sender and delete all copies of this message. Persistent Systems
Ltd. does not accept any liability for virus infected mails.


Mime
View raw message