hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian Fang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7240) Object store in HDFS
Date Fri, 05 Jun 2015 17:07:09 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14574840#comment-14574840
] 

Jian Fang commented on HDFS-7240:
---------------------------------

Sorry for jumping in. I saw the following statement and wonder why you need to follow S3FileSystem
closely. 

"OzoneFileSystem: Hadoop file system implementation on top of ozone, similar to S3FileSystem.
    * It will not support rename"

Not supporting rename could cause a lot of trouble because hadoop uses it to achieve some
thing like "two phase commit" in many places, for example, FileOutputCommitter. What would
you do for such use cases? Add extra logic to hadoop code or copy the data to the final destination?
The latter could be very expensive by the way. 

I am not very clear about the motivations here. Shouldn't some features have already been
covered by HBase? Would this feature make HDFS too fat and become difficult to manage? Or
is this on top of HDFS just like HBase? Also how do you handle small objects in the object
store?


> Object store in HDFS
> --------------------
>
>                 Key: HDFS-7240
>                 URL: https://issues.apache.org/jira/browse/HDFS-7240
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Jitendra Nath Pandey
>            Assignee: Jitendra Nath Pandey
>         Attachments: Ozone-architecture-v1.pdf
>
>
> This jira proposes to add object store capabilities into HDFS. 
> As part of the federation work (HDFS-1052) we separated block storage as a generic storage
layer. Using the Block Pool abstraction, new kinds of namespaces can be built on top of the
storage layer i.e. datanodes.
> In this jira I will explore building an object store using the datanode storage, but
independent of namespace metadata.
> I will soon update with a detailed design document.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message