hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghava Mutharaju <m.vijayaragh...@gmail.com>
Subject Re: appending to files on HDFS
Date Wed, 17 Feb 2010 01:30:58 GMT
Hi Eric,

       Thank you for your suggestions. I would do that and check it out.


Regards,
Raghava.

On Tue, Feb 16, 2010 at 8:00 PM, E. Sammer <eric@lifeless.net> wrote:

> On 2/16/10 7:54 PM, Raghava Mutharaju wrote:
>
>> Hi all,
>>
>>       I am using hadoop 0.20.1 version. I am trying to append some content
>> from a file on HDFS to another file on HDFS. I am using
>> FileSystem.append()
>> method to get the stream and then passed it to IOUtils.copyBytes() method.
>> I
>> got the following exception
>>
>>
>> org.apache.hadoop.ipc.RemoteException: java.io.IOException: Append to hdfs
>> not supported. Please refer to dfs.support.append configuration parameter.
>>
>>
>> I set the dfs.support.append to true in hdfs-site.xml. I restarted hadoop
>> (stop-all&  start-all) after this. Even after this, I am getting the above
>> exception.
>>
>> Where am I going wrong?
>>
>> Thank you for your suggestions.
>>
>> Regards,
>> Raghava.
>>
>>
> Raghava:
>
> In short, appends in HDFS are extremely experimental and dangerous. Most
> would advise you to leave this disabled. Your best option for "append" like
> behavior is to rewrite the file with new content being added at the end.
> Append support was briefly introduced and then removed as a number of issues
> came up. I believe the open (parent) JIRA issue tracking this is:
>
> http://issues.apache.org/jira/browse/HDFS-265
>
> Hope this helps.
> --
> Eric Sammer
> eric@lifeless.net
> http://esammer.blogspot.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message