hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-8210) Ozone: Implement storage container manager
Date Tue, 29 Mar 2016 21:53:25 GMT

     [ https://issues.apache.org/jira/browse/HDFS-8210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Chris Nauroth updated HDFS-8210:
    Attachment: HDFS-8210-HDFS-7240.006.patch

I'm attaching patch v006.  This refreshes the patch after the rebase of the feature branch
and also updates a few things based on ongoing testing:
# {{StorageContainerManager}} now provides a stub implementation of {{getStorageContainerLocations}}.
 The stub implementation simply reports all registered DataNodes as viable locations.  This
will evolve of course, but the stub is sufficient to enable some end-to-end testing of basic
service interactions.
# The partitioning code from previous revisions of this patch is not needed immediately and
thus not included.  As per above, partitioning will become important at a later phase of development
when {{getStorageContainerLocations}} is no longer a stub.
# I did a lot of internal refactoring for code reuse within {{StorageContainerManager}}. 
In particular, the constructor is much shorter now.
# I created {{MiniOzoneCluster}}, which starts a StorageContainerManager and multiple DataNodes,
suitable for writing tests.  Unlike {{MiniDFSCluster}}, it does not start a {{NameNode}}.
 I needed to make a small change in {{MiniDFSCluster}} to accommodate this.
# {{TestStorageContainerManager}} is a new test suite that covers basic service interactions
with DataNodes registering to the StorageContainerManager and clients then calling StorageContainerManager
to find container locations.

In addition to the new test suite, I tested manually by building a distro and running a StorageContainerManager,
DataNode, and client.

[~anu] or [~jnp], would you like to review?

> Ozone: Implement storage container manager 
> -------------------------------------------
>                 Key: HDFS-8210
>                 URL: https://issues.apache.org/jira/browse/HDFS-8210
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Jitendra Nath Pandey
>            Assignee: Chris Nauroth
>         Attachments: HDFS-8210-HDFS-7240.006.patch, HDFS-8210-HDFS-7240.1.patch, HDFS-8210-HDFS-7240.2.patch,
HDFS-8210-HDFS-7240.3.patch, HDFS-8210-HDFS-7240.4.patch, HDFS-8210-HDFS-7240.5.patch
> The storage container manager collects datanode heartbeats, manages replication and exposes
API to lookup containers. This jira implements storage container manager by re-using the block
manager implementation in namenode. This jira provides initial implementation that works with
datanodes. The additional protocols will be added in subsequent jiras.

This message was sent by Atlassian JIRA

View raw message