hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ishaaq Chandy <ish...@gmail.com>
Subject atomicity of copyFromLocal
Date Tue, 01 Mar 2011 04:22:24 GMT
Hi all,
How "atomic" is the copyFromLocal call? i.e. on process is in the midst of
uploading a file to HDFS is it possible for another process to start reading
it before the upload is complete?

I am currently safeguarding my code from this possibility by uploading it to
a temporary directory and the renaming it to its final destination (the
assumption being that a rename is "more atomic" than copyFromLocal), but I'd
like to avoid doing this in two steps if possible.


View raw message