hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stuti Awasthi <stutiawas...@hcl.com>
Subject File Integrity in HDFS
Date Tue, 01 May 2012 09:52:11 GMT
Hi All,

I have a scenario in which Input files are copied to HDFS and MR jobs run on the input directory.
Now there can be a scenario in which file is getting copied to HDFS and MR jobs starts , in
this case I do not want my MR job to pick those files which are getting copied to hdfs and
process of copying is not complete.

Is there any way/api to check that if the file is not completely written to HDFS we can know.

Stuti Awasthi
HCL Comnet Systems and Services Ltd
F-8/9 Basement, Sec-3,Noida.



The contents of this e-mail and any attachment(s) are confidential and intended for the named
recipient(s) only.
E-mail transmission cannot be guaranteed to be secure or error-free as information could be
intercepted, corrupted,
lost, destroyed, arrive late or incomplete, or contain viruses.The e mail and its contents
(with or without referred
errors) shall therefore not attach any liability on the originator or HCL or its affiliates.
Any views or opinions
presented in this email are solely those of the author and may not necessarily reflect the
opinions of HCL or its
affiliates. Any form of reproduction, dissemination, copying, disclosure, Modification, distribution
and/or publication
of this message without the prior written consent of the author of this e-mail is strictly
prohibited. If you have
received this email in error please delete it and notify the sender immediately. Before opening
any mail and attachments
please check them for viruses and defect.


View raw message