Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Mon, 9 Apr 2012 18:27:19 +0000 (UTC)
From: "Todd Lipcon (Commented) (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: 
 <187926904.3813.1333996039495.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <192703014.24915.1333738163586.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HDFS-3222) DFSInputStream#openInfo should not
 silently get the length as 0 when locations length is zero for last partial
 block.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-3222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13250032#comment-13250032 ] 

Todd Lipcon commented on HDFS-3222:
-----------------------------------

bq. It may be difficult for the clients to differ whether this is real corruption or it will be recovered after DN reports to NN.

What's the difference? If none of the DNs holding a block have reported a replica, it's missing/corrupt. The same is true of finalized blocks - if three DNs crash, and we have no replicas anymore, it still might come back if an admin fixes one of the DNs.

bq. you mean reader will pass the option? (a) or (b).

Sorry, I wasn't clear. Right now, the behavior is that, when we call open() on a file which is under construction, we always go to the DNs holding the last block to find the length. My proposal is the following:
- on open(), do not determine the visible length of the file. Set the member variable to something like "-1" to indicate it's still unknown
- in the code that opens a block reader, change it to check if it's about to read from the last block. If it is, try to determine the visible length.
- in the explicit getVisibleLength() call, if it's not determined yet, try to determine the visible length

With the above changes, we can allow a client who only wants to access the first blocks of a file to do so without having to contact the DNs holding the last block. But as soon as the client wants to access the under-construction block, or explicitly wants to know the visible length, then we go to the DNs.

                
> DFSInputStream#openInfo should not silently get the length as 0 when locations length is zero for last partial block.
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3222
>                 URL: https://issues.apache.org/jira/browse/HDFS-3222
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client
>    Affects Versions: 1.0.3, 2.0.0, 3.0.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>         Attachments: HDFS-3222-Test.patch
>
>
> I have seen one situation with Hbase cluster.
> Scenario is as follows:
> 1)1.5 blocks has been written and synced.
> 2)Suddenly cluster has been restarted.
> Reader opened the file and trying to get the length., By this time partial block contained DNs are not reported to NN. So, locations for this partial block would be 0. In this case, DFSInputStream assumes that, 1 block size as final size.
> But reader also assuming that, 1 block size is the final length and setting his end marker. Finally reader ending up reading only partial data. Due to this, HMaster could not replay the complete edits. 
> Actually this happend with 20 version. Looking at the code, same should present in trunk as well.
> {code}
>     int replicaNotFoundCount = locatedblock.getLocations().length;
>     
>     for(DatanodeInfo datanode : locatedblock.getLocations()) {
> ..........
> ..........
>  // Namenode told us about these locations, but none know about the replica
>     // means that we hit the race between pipeline creation start and end.
>     // we require all 3 because some other exception could have happened
>     // on a DN that has it.  we want to report that error
>     if (replicaNotFoundCount == 0) {
>       return 0;
>     }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira