hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-2288) Replicas awaiting recovery should return a full visible length
Date Tue, 13 Sep 2011 03:08:09 GMT

    [ https://issues.apache.org/jira/browse/HDFS-2288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13103296#comment-13103296

Konstantin Shvachko commented on HDFS-2288:

Todd, visible length is a notion which exists when pipeline is in progress. As Nicholas said
it guarantees consistency of reads from replicas at different stages.
In your scenario if I understood it correctly after writing one replica and failing on hflush(),
nobody would've seen any bytes yet, therefore visible length should be 0.
When DN restarts after the crash there is no visible length since there is no pipeline. Non-finalized
replicas become RWR, which means the recovery hasn't started for them yet. And clients should
not read from RWR replicas until the recovery finishes.
Section 8.3 of the design doc says "The length of an rwr is a negative number." Also worth
checking the replica state transition diagram under 9.1
So I think the behavior is correct (to the specs). What is the HBase logic there?

> Replicas awaiting recovery should return a full visible length
> --------------------------------------------------------------
>                 Key: HDFS-2288
>                 URL: https://issues.apache.org/jira/browse/HDFS-2288
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Critical
>             Fix For: 0.23.0
>         Attachments: hdfs-2288.txt
> Currently, if the client calls getReplicaVisibleLength for a RWR, it returns a visible
length of 0. This causes one of HBase's tests to fail, and I believe it's incorrect behavior.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message