hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amir Langer (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-5731) Refactoring to define interfaces between BM and NN and simplify the flow between them
Date Mon, 13 Jan 2014 06:01:50 GMT

     [ https://issues.apache.org/jira/browse/HDFS-5731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Amir Langer updated HDFS-5731:
------------------------------

    Description: 
Start the separation of BlockManager (BM) from NameNode (NN) by simplifying the flow between
the two components and defining API interfaces between them.
Logic to calls from Datanodes should be in the BM.
NN should interact with BM using few calls and BM should use the return types as much as possible
to pass information to the NN.
APIs between them should be defined as interfaces so later it can be improved to not use the
same object instances and turned into a real protocol.
The emphasis is on restructuring the request execution flows between the NN and BM in a way
that will minimize the latency increase when the BM implementation becomes remote. Namely,
the API flows are restructured in a way that BM is called at most once per request. 
The two components (NN and BM) still exist in the same VM and share the same memory space.
There is still a 1:1 relationship  between them. 
This task should maintain backward compatibility


  was:
Start the separation of BlockManager (BM) from NameNode (NN) by simplifying the flow between
the two components and defining API interfaces between them.
The emphasis is on simplifying the transactional flows between the NN and BM and make sure
there will be no more than one call between them.  
The two components still exist in the same VM and use the same memory space (using the same
instances).
Logic to calls from Datanodes should be in the BM.
NN should interact with BM using few calls and BM should use the return types as much as possible
to pass information to the NN.
APIs between them should be defined as interfaces so later it can be improved to not use the
same object instances and turned into a real protocol.
This still assumes a one to one relation between NN and BM, same VM and does not handle lifecycle
of the service.

This task should maintain backward compatibility



> Refactoring to define interfaces between BM and NN and simplify the flow between them
> -------------------------------------------------------------------------------------
>
>                 Key: HDFS-5731
>                 URL: https://issues.apache.org/jira/browse/HDFS-5731
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: namenode
>            Reporter: Amir Langer
>         Attachments: 0001-Separation-of-BM-from-NN-Step1-introduce-APIs-as-int.patch
>
>
> Start the separation of BlockManager (BM) from NameNode (NN) by simplifying the flow
between the two components and defining API interfaces between them.
> Logic to calls from Datanodes should be in the BM.
> NN should interact with BM using few calls and BM should use the return types as much
as possible to pass information to the NN.
> APIs between them should be defined as interfaces so later it can be improved to not
use the same object instances and turned into a real protocol.
> The emphasis is on restructuring the request execution flows between the NN and BM in
a way that will minimize the latency increase when the BM implementation becomes remote. Namely,
the API flows are restructured in a way that BM is called at most once per request. 
> The two components (NN and BM) still exist in the same VM and share the same memory space.
> There is still a 1:1 relationship  between them. 
> This task should maintain backward compatibility



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message