hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: HDFS file append
Date Tue, 15 Jun 2010 15:24:29 GMT
On Tue, Jun 15, 2010 at 5:31 AM, Jan Stöcker <Jan.Stoecker@q2web.de> wrote:

> Hello,
>
>
>
> in the hdfs-default.xml, I found the warning that the property
>
> "dfs.support.append" is false by default, because of the "append
>
> code" containing bugs.
>
> For our test case (which is not very complicated),  I tried it out
>
> nevertheless (in a C program using libhdfs), and I did not
>
> encounter any problems yet. The only thing I noticed is that the
>
> modification date of the file to which I append is not changed.
>
>
>
> Could someone indicate which other bugs may occur? And in
>
> which cases? Because the append could be quite useful for me,
>
> and I would like to know the risk.
>

Mostly in failure handling cases, but not entirely. The result is usually
truncated files, sometimes truncated as if your append didn't happen,
sometimes the entire last block going missing. Look for the 0.20-append fix
version on JIRA for a more significantly list.

I'd recommend waiting a couple weeks and using the hadoop-0.20-append branch
which will contain fixes for many of these bugs.

-Todd


-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message