hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anu Engineer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10897) Ozone: SCM: Add NodeManager
Date Fri, 23 Sep 2016 18:23:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517176#comment-15517176
] 

Anu Engineer commented on HDFS-10897:
-------------------------------------

I have been told that ozone JIRAs need a little more context at times for people to understand
what is going on. Please feel free to ask more clarifying questions as needed.

Background: 

SCM in HDFS-7240 branch replaces Namenode and uses the same datanode protocol to talk to datanode.
This was a hack to make ozone simulate an end-to-end scenario.  The ultimate goal is to have
ozone work in an HDFS cluster with no impact to HDFS. Hence a series of JIRAs will attempt
to create a new SCM.

These JIRAs collectively will try to do the following.

* Create a new SCM datanode protocol that takes care of SCM heartbeats and container reports.
* Create a new SCM client protocol that will allow clients to manage SCM.
* Provide SCM APIs needed by KSM.
* Make sure that MiniOzoneCluster runs both Namenode and SCM concurrently.
* Make sure that Ozone can be enabled and disabled by a simple ozone setting with no impact
to HDFS.

Since these changes can be pretty huge, in the order of 100s of KBs of code,  We are going
to break them up into a series of small code changes that make it easy to code review.

The plan is add required components on both SCM (server side) and container (Datanode side).
Then we will bring all these changes together by replacing the StorageContainerManager class
that will expose the network endpoints. Then we will go on to enable heartbeats and container
reports. This is to make sure that we don't break the existing unit tests of ozone as the
work is coming together.

To make it easy to track these changes, all these JIRAs are prefixed with Ozone: SCM:. Also
a good way to code review this change might be to first read the tests, they will explain
what are the expectations of the node manager.

> Ozone: SCM: Add NodeManager
> ---------------------------
>
>                 Key: HDFS-10897
>                 URL: https://issues.apache.org/jira/browse/HDFS-10897
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>    Affects Versions: HDFS-7240
>            Reporter: Anu Engineer
>            Assignee: Anu Engineer
>
> Add a nodeManager class that will be used by Storage Controller Manager eventually.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message