hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1142) Lease recovery doesn't reassign lease when triggered by append()
Date Tue, 18 May 2010 05:46:56 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12868541#action_12868541
] 

Todd Lipcon commented on HDFS-1142:
-----------------------------------

bq. The test assumes that hflush() should not succeed after the recovery started, but this
assumption is incorrect.

HBase is using contracts like this to do IO fencing during regionserver recovery operations
(eg see discussion on HBASE-2231 and HBASE-2238). The ability to use lease recovery to lock
out a writer whose soft lease has expired provides lock-like functionality tied to the storage.
Without HDFS itself providing a file locking primitive, it becomes essentially impossible
to do correct recovery in systems like HBase where an old writer needs to be forcibly disallowed
from continued progress. Using an external system like ZK since it is asynchronous in nature.

Let me think if we have an alternate route to the same kind of behavior using the semantics
you're describing.

> Lease recovery doesn't reassign lease when triggered by append()
> ----------------------------------------------------------------
>
>                 Key: HDFS-1142
>                 URL: https://issues.apache.org/jira/browse/HDFS-1142
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: name-node
>    Affects Versions: 0.21.0
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Blocker
>         Attachments: hdfs-1142.txt, hdfs-1142.txt
>
>
> If a soft lease has expired and another writer calls append(), it triggers lease recovery
but doesn't reassign the lease to a new owner. Therefore, the old writer can continue to allocate
new blocks, try to steal back the lease, etc. This is for the testRecoveryOnBlockBoundary
case of HDFS-1139

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message