hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yongjun Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7342) Lease Recovery doesn't happen some times
Date Wed, 26 Nov 2014 20:19:14 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14226765#comment-14226765
] 

Yongjun Zhang commented on HDFS-7342:
-------------------------------------

Hi [~vinayrpet],

Thanks for further explanation. I studied your latest two comments, and I agree with your
analysis: the case that penultimate block being in COMMITTEED sate with minimal replication
met may not exist. In other words, when penultimate block is in COMMITTED state, the minimal
replication is not met.

Going back to the source that triggerd this discussion, it's the scenario#1 described in:
 https://issues.apache.org/jira/browse/HDFS-4882?focusedCommentId=14213992&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14213992

Here is what I think now:

scenario#1 exists for last block but not penultimate block
scenarion#2 exists for both penultimate and last block

My suggested solution for this jia is to share the same code that handles the case when last
block is COMMITTEED, with some modifications:

* For scenarion#1 (last block), my suggested solution is to forceComplete the block before
calling  {{finalizeINodeFileUnderConstruction}}:
{code}
        if (lastBlockState == BlockUCState.COMMITTED) {
          getBlockManager().forceCompleteBlock(pendingFile,
              (BlockInfoUnderConstruction) lastBlock);
        }
        finalizeINodeFileUnderConstruction(src, pendingFile,
            iip.getLatestSnapshotId());
{code}
thus avoiding the exception thrown from {{finalizeINodeFileUnderConstruction}} caused by
{code}
Preconditions.checkState(blocks[i].isComplete(), "Failed to finalize"
          + " %s %s since blocks[%s] is non-complete, where blocks=%s.",
          getClass().getSimpleName(), this, i, Arrays.asList(blocks));
{code}

* For scenarion#2, the same code can be shared to handle the case that when penultimate block
is COMMITTED with minimal replication NOT met. the lease will be recovered, even if it's this
kind of state.

On top of this suggested solution, we can add what Kihwal suggested.

What do you think?

Thanks.


> Lease Recovery doesn't happen some times
> ----------------------------------------
>
>                 Key: HDFS-7342
>                 URL: https://issues.apache.org/jira/browse/HDFS-7342
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.0.0-alpha
>            Reporter: Ravi Prakash
>            Assignee: Ravi Prakash
>         Attachments: HDFS-7342.1.patch, HDFS-7342.2.patch, HDFS-7342.3.patch
>
>
> In some cases, LeaseManager tries to recover a lease, but is not able to. HDFS-4882 describes
a possibility of that. We should fix this



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message