hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jitendra Nath Pandey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7240) Object store in HDFS
Date Tue, 30 Jun 2015 21:25:12 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14609099#comment-14609099

Jitendra Nath Pandey commented on HDFS-7240:

  For prefix scans we are planning to support range partitioned index in the storage container
manager. I will update the design document. That will allow users to have hierarchy under
the bucket.

     The storage volumes do not really change the semantics that much from S3. For a url like
{{http://host:port/volume/bucket/key}}, apart from the volume prefix, the semantics are really
similar. I believe, having a notion of 'admin created volumes' allows us to have buckets which
are very similar to S3, as it provides clear domains of admin control vs user control. Volume
can be viewed as accounts, which are kind of similar to accounts in S3 or Azure.
     It will be a bigger deviation if we insist that only admin can create buckets. It is
useful to organize data in buckets using prefix, but having that as the only mechanism for
organizing data seems too restrictive.
     For similarity to S3, we will try to have similar headers and auth semantics as well,
which are also very important for portability.

> Object store in HDFS
> --------------------
>                 Key: HDFS-7240
>                 URL: https://issues.apache.org/jira/browse/HDFS-7240
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Jitendra Nath Pandey
>            Assignee: Jitendra Nath Pandey
>         Attachments: Ozone-architecture-v1.pdf
> This jira proposes to add object store capabilities into HDFS. 
> As part of the federation work (HDFS-1052) we separated block storage as a generic storage
layer. Using the Block Pool abstraction, new kinds of namespaces can be built on top of the
storage layer i.e. datanodes.
> In this jira I will explore building an object store using the datanode storage, but
independent of namespace metadata.
> I will soon update with a detailed design document.

This message was sent by Atlassian JIRA

View raw message