hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException:'
Date Mon, 02 Apr 2012 13:52:30 GMT

What does your job do? Create files directly on HDFS? If so, do you
follow this method?:

A local filesystem may not complain if you re-create an existent file.
HDFS' behavior here is different. This simple Python test is what I
>>> a = open('a', 'w')
>>> a.write('f')
>>> b = open('a', 'w')
>>> b.write('s')
>>> a.close(), b.close()
>>> open('a').read()

Hence it is best to use the FileOutputCommitter framework as detailed
in the mentioned link.

On Mon, Apr 2, 2012 at 7:09 PM, Jay Vyas <jayunit100@gmail.com> wrote:
> Hi guys:
> I have a map reduce job that runs normally on local file system from
> eclipse, *but* it fails on HDFS running in psuedo distributed mode.
> The exception I see is
> *org.apache.hadoop.ipc.RemoteException:
> org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException:*
> Any thoughts on why this might occur in psuedo distributed mode, but not in
> regular file system ?

Harsh J

View raw message