hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Stöcker <Jan.Stoec...@q2web.de>
Subject AW: HDFS file append
Date Wed, 16 Jun 2010 06:57:11 GMT
O.k., thanks for the information & the suggestions!

Jan

-----Ursprüngliche Nachricht-----
Von: Todd Lipcon [mailto:todd@cloudera.com] 
Gesendet: Dienstag, 15. Juni 2010 18:55
An: general@hadoop.apache.org
Betreff: Re: HDFS file append

On Tue, Jun 15, 2010 at 9:53 AM, Dhruba Borthakur <dhruba@gmail.com> wrote:

> You can start downloading and using the code form the hadoop-0.20-append.
>
>
Though it's still missing some patches, so I'd recommend SVN checkouts and
watching the branch :)

-Todd


> thanks,
> dhruba
>
> On Tue, Jun 15, 2010 at 8:24 AM, Todd Lipcon <todd@cloudera.com> wrote:
>
> > On Tue, Jun 15, 2010 at 5:31 AM, Jan Stöcker <Jan.Stoecker@q2web.de>
> > wrote:
> >
> > > Hello,
> > >
> > >
> > >
> > > in the hdfs-default.xml, I found the warning that the property
> > >
> > > "dfs.support.append" is false by default, because of the "append
> > >
> > > code" containing bugs.
> > >
> > > For our test case (which is not very complicated),  I tried it out
> > >
> > > nevertheless (in a C program using libhdfs), and I did not
> > >
> > > encounter any problems yet. The only thing I noticed is that the
> > >
> > > modification date of the file to which I append is not changed.
> > >
> > >
> > >
> > > Could someone indicate which other bugs may occur? And in
> > >
> > > which cases? Because the append could be quite useful for me,
> > >
> > > and I would like to know the risk.
> > >
> >
> > Mostly in failure handling cases, but not entirely. The result is usually
> > truncated files, sometimes truncated as if your append didn't happen,
> > sometimes the entire last block going missing. Look for the 0.20-append
> fix
> > version on JIRA for a more significantly list.
> >
> > I'd recommend waiting a couple weeks and using the hadoop-0.20-append
> > branch
> > which will contain fixes for many of these bugs.
> >
> > -Todd
> >
> >
> > --
> > Todd Lipcon
> > Software Engineer, Cloudera
> >
>
>
>
> --
> Connect to me at http://www.facebook.com/dhruba
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
View raw message