hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8210) Ozone: Implement storage container manager
Date Tue, 29 Mar 2016 21:53:25 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Nauroth updated HDFS-8210:
--------------------------------
    Attachment: HDFS-8210-HDFS-7240.006.patch

I'm attaching patch v006.  This refreshes the patch after the rebase of the feature branch
and also updates a few things based on ongoing testing:
# {{StorageContainerManager}} now provides a stub implementation of {{getStorageContainerLocations}}.
 The stub implementation simply reports all registered DataNodes as viable locations.  This
will evolve of course, but the stub is sufficient to enable some end-to-end testing of basic
service interactions.
# The partitioning code from previous revisions of this patch is not needed immediately and
thus not included.  As per above, partitioning will become important at a later phase of development
when {{getStorageContainerLocations}} is no longer a stub.
# I did a lot of internal refactoring for code reuse within {{StorageContainerManager}}. 
In particular, the constructor is much shorter now.
# I created {{MiniOzoneCluster}}, which starts a StorageContainerManager and multiple DataNodes,
suitable for writing tests.  Unlike {{MiniDFSCluster}}, it does not start a {{NameNode}}.
 I needed to make a small change in {{MiniDFSCluster}} to accommodate this.
# {{TestStorageContainerManager}} is a new test suite that covers basic service interactions
with DataNodes registering to the StorageContainerManager and clients then calling StorageContainerManager
to find container locations.

In addition to the new test suite, I tested manually by building a distro and running a StorageContainerManager,
DataNode, and client.

[~anu] or [~jnp], would you like to review?


> Ozone: Implement storage container manager 
> -------------------------------------------
>
>                 Key: HDFS-8210
>                 URL: https://issues.apache.org/jira/browse/HDFS-8210
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Jitendra Nath Pandey
>            Assignee: Chris Nauroth
>         Attachments: HDFS-8210-HDFS-7240.006.patch, HDFS-8210-HDFS-7240.1.patch, HDFS-8210-HDFS-7240.2.patch,
HDFS-8210-HDFS-7240.3.patch, HDFS-8210-HDFS-7240.4.patch, HDFS-8210-HDFS-7240.5.patch
>
>
> The storage container manager collects datanode heartbeats, manages replication and exposes
API to lookup containers. This jira implements storage container manager by re-using the block
manager implementation in namenode. This jira provides initial implementation that works with
datanodes. The additional protocols will be added in subsequent jiras.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message