hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5731) Refactoring to define interfaces between BM and NN and simplify the flow between them
Date Tue, 04 Feb 2014 19:32:18 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13891067#comment-13891067

Daryn Sharp commented on HDFS-5731:

I'm trying to review this patch but it's a bit too much.  Broad API and functional changes
across the NN is difficult to verify for correctness.

To expedite the review, this patch should be broken into multiple jiras where possible.  Each
jira should encompass one specific area of change.  Examples include jiras for the changes
to FSNamesystem's verifyReplication, getBlockLocations, commitBlockSynchronization, jmx changes,

Generally, api changes should accompany the patch for which they are required.  The intent/reason
for each change will be clear to the reviewer.

> Refactoring to define interfaces between BM and NN and simplify the flow between them
> -------------------------------------------------------------------------------------
>                 Key: HDFS-5731
>                 URL: https://issues.apache.org/jira/browse/HDFS-5731
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>            Reporter: Amir Langer
>         Attachments: 0001-Separation-of-BM-from-NN-Step1-introduce-APIs-as-int.patch
> Start the separation of BlockManager (BM) from NameNode (NN) by simplifying the flow
between the two components and defining API interfaces between them. This is done to enable
future transformation into a clean RPC protocol.  
> Logic to calls from Datanodes should be in the BM.
> NN should interact with BM using few calls and BM should use the return types as much
as possible to pass information to the NN.
> The emphasis is on restructuring the request execution flows between the NN and BM in
a way that will minimize the latency increase when the BM implementation becomes remote. Namely,
the API flows are restructured in a way that BM is called at most once per request. 
> The two components (NN and BM) still exist in the same VM and share the same memory space.
> NN and BM share the same lifecycle – it is assumed that they can't crash/restart separately.

> There is still a 1:1 relationship  between them. 
> APIs between NN and BM will be improved to not use the same object instances and turned
into a real protocol.
> This task should maintain backward compatibility

This message was sent by Atlassian JIRA

View raw message