hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3222) DFSInputStream#openInfo should not silently get the length as 0 when locations length is zero for last partial block.
Date Fri, 20 Apr 2012 19:24:37 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13258493#comment-13258493
] 

Hadoop QA commented on HDFS-3222:
---------------------------------

-1 overall.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12523528/HDFS-3222.patch
  against trunk revision .

    +1 @author.  The patch does not contain any @author tags.

    +1 tests included.  The patch appears to include 1 new or modified test files.

    +1 javadoc.  The javadoc tool did not generate any warning messages.

    +1 javac.  The applied patch does not increase the total number of javac compiler warnings.

    +1 eclipse:eclipse.  The patch built with eclipse:eclipse.

    -1 findbugs.  The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

    +1 release audit.  The applied patch does not increase the total number of release audit
warnings.

    +1 core tests.  The patch passed unit tests in .

    +1 contrib tests.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/2310//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/2310//artifact/trunk/hadoop-hdfs-project/patchprocess/newPatchFindbugsWarningshadoop-hdfs.html
Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2310//console

This message is automatically generated.
                
> DFSInputStream#openInfo should not silently get the length as 0 when locations length
is zero for last partial block.
> ---------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-3222
>                 URL: https://issues.apache.org/jira/browse/HDFS-3222
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs client
>    Affects Versions: 1.0.3, 2.0.0, 3.0.0
>            Reporter: Uma Maheswara Rao G
>            Assignee: Uma Maheswara Rao G
>         Attachments: HDFS-3222-Test.patch, HDFS-3222.patch
>
>
> I have seen one situation with Hbase cluster.
> Scenario is as follows:
> 1)1.5 blocks has been written and synced.
> 2)Suddenly cluster has been restarted.
> Reader opened the file and trying to get the length., By this time partial block contained
DNs are not reported to NN. So, locations for this partial block would be 0. In this case,
DFSInputStream assumes that, 1 block size as final size.
> But reader also assuming that, 1 block size is the final length and setting his end marker.
Finally reader ending up reading only partial data. Due to this, HMaster could not replay
the complete edits. 
> Actually this happend with 20 version. Looking at the code, same should present in trunk
as well.
> {code}
>     int replicaNotFoundCount = locatedblock.getLocations().length;
>     
>     for(DatanodeInfo datanode : locatedblock.getLocations()) {
> ..........
> ..........
>  // Namenode told us about these locations, but none know about the replica
>     // means that we hit the race between pipeline creation start and end.
>     // we require all 3 because some other exception could have happened
>     // on a DN that has it.  we want to report that error
>     if (replicaNotFoundCount == 0) {
>       return 0;
>     }
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message