Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Date: Mon, 13 Jan 2014 06:01:50 +0000 (UTC)
From: "Amir Langer (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12687857.1389189814781.85357.1389592910149@arcas>
In-Reply-To: <JIRA.12687857.1389189814781@arcas>
References: <JIRA.12687857.1389189814781@arcas>
Subject: [jira] [Updated] (HDFS-5731) Refactoring to define interfaces
 between BM and NN and simplify the flow between them
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/HDFS-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Amir Langer updated HDFS-5731:
------------------------------

    Description: 
Start the separation of BlockManager (BM) from NameNode (NN) by simplifying the flow between the two components and defining API interfaces between them.
Logic to calls from Datanodes should be in the BM.
NN should interact with BM using few calls and BM should use the return types as much as possible to pass information to the NN.
APIs between them should be defined as interfaces so later it can be improved to not use the same object instances and turned into a real protocol.
The emphasis is on restructuring the request execution flows between the NN and BM in a way that will minimize the latency increase when the BM implementation becomes remote. Namely, the API flows are restructured in a way that BM is called at most once per request. 
The two components (NN and BM) still exist in the same VM and share the same memory space.
There is still a 1:1 relationship  between them. 
This task should maintain backward compatibility


  was:
Start the separation of BlockManager (BM) from NameNode (NN) by simplifying the flow between the two components and defining API interfaces between them.
The emphasis is on simplifying the transactional flows between the NN and BM and make sure there will be no more than one call between them.  
The two components still exist in the same VM and use the same memory space (using the same instances).
Logic to calls from Datanodes should be in the BM.
NN should interact with BM using few calls and BM should use the return types as much as possible to pass information to the NN.
APIs between them should be defined as interfaces so later it can be improved to not use the same object instances and turned into a real protocol.
This still assumes a one to one relation between NN and BM, same VM and does not handle lifecycle of the service.

This task should maintain backward compatibility


> Refactoring to define interfaces between BM and NN and simplify the flow between them
> -------------------------------------------------------------------------------------
>
>                 Key: HDFS-5731
>                 URL: https://issues.apache.org/jira/browse/HDFS-5731
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>            Reporter: Amir Langer
>         Attachments: 0001-Separation-of-BM-from-NN-Step1-introduce-APIs-as-int.patch
>
>
> Start the separation of BlockManager (BM) from NameNode (NN) by simplifying the flow between the two components and defining API interfaces between them.
> Logic to calls from Datanodes should be in the BM.
> NN should interact with BM using few calls and BM should use the return types as much as possible to pass information to the NN.
> APIs between them should be defined as interfaces so later it can be improved to not use the same object instances and turned into a real protocol.
> The emphasis is on restructuring the request execution flows between the NN and BM in a way that will minimize the latency increase when the BM implementation becomes remote. Namely, the API flows are restructured in a way that BM is called at most once per request. 
> The two components (NN and BM) still exist in the same VM and share the same memory space.
> There is still a 1:1 relationship  between them. 
> This task should maintain backward compatibility


--
This message was sent by Atlassian JIRA
(v6.1.5#6160)