hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jim Kellerman (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5744) Revisit append
Date Mon, 18 May 2009 23:06:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710530#action_12710530

Jim Kellerman commented on HADOOP-5744:

> Hairong Kuang added a comment - 18/May/09 03:53 PM
> Note that the current implementation in 0.20 does not provide the second guarantee described
> A byte saw by a reader become invisible when the replica it read from fails. Even when
the replica that
> a reader is reading from is alive, the recovery of an error caused by another replica
in the pipeline may
> remove a byte saw by a reader temporary from dfs but recover the byte later. A removed
byte may never
> get recovered if the writer dies or the writer does not have this byte any more. 

Yes, this is what HADOOP-4379 is trying to address with Doug Cutting's comment:

> Doug Cutting added a comment - 26/Jan/09 11:20 AM
>I think it would be better to add a SequenceFile.Writer method that flushes and calls
> FSDataOutputStream.sync. The problem is that SequenceFile.Writer#sync() should
> probably be called flush(), not sync(), but that'd be hard to change. So perhaps we
> should add a SequenceFile.Writer#syncFs() method?

> Revisit append
> --------------
>                 Key: HADOOP-5744
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5744
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.20.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.21.0
>         Attachments: AppendSpec.pdf
> HADOOP-1700 and related issues have put a lot of efforts to provide the first implementation
of append. However, append is such a complex feature. It turns out that there are issues that
were initially seemed trivial but needs a careful design. This jira revisits append, aiming
for a design and implementation supporting a semantics that are acceptable to its users.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message