hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dhruba Borthakur <dhr...@gmail.com>
Subject Re: HDFS file append
Date Tue, 15 Jun 2010 16:53:22 GMT
You can start downloading and using the code form the hadoop-0.20-append.

thanks,
dhruba

On Tue, Jun 15, 2010 at 8:24 AM, Todd Lipcon <todd@cloudera.com> wrote:

> On Tue, Jun 15, 2010 at 5:31 AM, Jan Stöcker <Jan.Stoecker@q2web.de>
> wrote:
>
> > Hello,
> >
> >
> >
> > in the hdfs-default.xml, I found the warning that the property
> >
> > "dfs.support.append" is false by default, because of the "append
> >
> > code" containing bugs.
> >
> > For our test case (which is not very complicated),  I tried it out
> >
> > nevertheless (in a C program using libhdfs), and I did not
> >
> > encounter any problems yet. The only thing I noticed is that the
> >
> > modification date of the file to which I append is not changed.
> >
> >
> >
> > Could someone indicate which other bugs may occur? And in
> >
> > which cases? Because the append could be quite useful for me,
> >
> > and I would like to know the risk.
> >
>
> Mostly in failure handling cases, but not entirely. The result is usually
> truncated files, sometimes truncated as if your append didn't happen,
> sometimes the entire last block going missing. Look for the 0.20-append fix
> version on JIRA for a more significantly list.
>
> I'd recommend waiting a couple weeks and using the hadoop-0.20-append
> branch
> which will contain fixes for many of these bugs.
>
> -Todd
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>



-- 
Connect to me at http://www.facebook.com/dhruba

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message