hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5744) Revisit append
Date Tue, 19 May 2009 00:04:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12710547#action_12710547

Hairong Kuang commented on HADOOP-5744:

Jim/Stack, thanks for your comments. This makes us to understand our users' requirements better.

> It doesn't have to be closed as long the reader is able to go all the ways up to the
last sync made by the writer before crash.
Yes, all flushed data are guaranteed to be read by a new reader. I also want you to be aware
that the reader may read beyond the flushed data.

> The file may have more bytes than when it was previously read. This is a norm case. Will
this be an issue to hbase? How would this circumstance arise?
Our idea is that not all the data received by a datanode are visible to a user. For example.
if a dfs application calls flush, the data left the client machine and are being pushed to
all datanodes, but the application dies before it receives the flush ack. This flush is considered
as a failure. The flushed data may have reached none, one, or all datanodes. So the data is
not guaranteed to be visible to a reader. But the lease recovery MAY still keep the data in
the file if the data have reached one or all datanodes.

> Revisit append
> --------------
>                 Key: HADOOP-5744
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5744
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: dfs
>    Affects Versions: 0.20.0
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.21.0
>         Attachments: AppendSpec.pdf
> HADOOP-1700 and related issues have put a lot of efforts to provide the first implementation
of append. However, append is such a complex feature. It turns out that there are issues that
were initially seemed trivial but needs a careful design. This jira revisits append, aiming
for a design and implementation supporting a semantics that are acceptable to its users.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message