hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5682) Heterogeneous Storage phase 2 - APIs to expose Storage Types
Date Fri, 27 Mar 2015 19:37:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14384471#comment-14384471

Arpit Agarwal commented on HDFS-5682:

Most of the remaining work planned for this Jira has been completed under HDFS-6584 and HDFS-7584.
There is also some overlap with HDFS-6581. I plan to resolve this Jira as obsolete soon.

Before you start trying to define multiple types of storage, you need to first consider the
data flow in and out and what sort of processes occur on the storage and then for each type
of storage, how you would implement it.
As an example... What happens to the data when there's an insert, compaction, major compaction,
indexing (secondary) , etc ... and for each of these categories how the new storage unit will
Hi [~msegel], none of these are filesystem primitives. HDFS just makes raw storage types available
for use. The kind of analysis you describe is relevant to components higher up in the stack.

> Heterogeneous Storage phase 2 - APIs to expose Storage Types
> ------------------------------------------------------------
>                 Key: HDFS-5682
>                 URL: https://issues.apache.org/jira/browse/HDFS-5682
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: datanode, namenode
>            Reporter: Arpit Agarwal
>            Assignee: Arpit Agarwal
>         Attachments: 20140522-Heterogeneous-Storages-API.pdf
> Phase 1 (HDFS-2832) added support to present the DataNode as a collection of discrete
storages of different types.
> This Jira is to track phase 2 of the Heterogeneous Storage work which involves exposing
Storage Types to applications and adding Quota Management support for administrators.
> This phase will also include tools support for administrators/users.

This message was sent by Atlassian JIRA

View raw message