hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Gray (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1520) HDFS 20 append: Lightweight NameNode operation to trigger lease recovery
Date Tue, 30 Nov 2010 00:01:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12965012#action_12965012
] 

Jonathan Gray commented on HDFS-1520:
-------------------------------------

In HBase, we may want to be able to prevent the existing writer from doing any further writing.
 Not sure how exactly this relates to that but if we had that option on log splitting,  we'd
use it as we do not expect writes during the log split.

On the replay, we might want to actually tail the split up logs while the master splits them
(not now but maybe in the future), so we wouldn't want to prevent concurrent writing in that
case.

> HDFS 20 append: Lightweight NameNode operation to trigger lease recovery
> ------------------------------------------------------------------------
>
>                 Key: HDFS-1520
>                 URL: https://issues.apache.org/jira/browse/HDFS-1520
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>    Affects Versions: 0.20-append
>            Reporter: Hairong Kuang
>            Assignee: Hairong Kuang
>             Fix For: 0.20-append
>
>
> Currently HBase uses append to trigger the close of HLog during Hlog split. Append is
a very expensive operation, which involves not only NameNode operations but creating a writing
pipeline. If one of datanodes on the pipeline has a problem, this recovery may takes minutes.
I'd like implement a lightweight NameNode operation to trigger lease recovery and make HBase
to use this instead.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message