Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 22 Nov 2017 22:42:01 +0000 (UTC)
From: "Aaron Fabbri (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Message-ID: <JIRA.12747878.1413243276000.304643.1511390521650@Atlassian.JIRA>
In-Reply-To: <JIRA.12747878.1413243276000@Atlassian.JIRA>
References: <JIRA.12747878.1413243276000@Atlassian.JIRA> <JIRA.12747878.1413243276395@jira-lw-us.apache.org>
Subject: [jira] [Commented] (HDFS-7240) Object store in HDFS
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 22 Nov 2017 22:42:06 -0000


    [ https://issues.apache.org/jira/browse/HDFS-7240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16263485#comment-16263485 ] 

Aaron Fabbri commented on HDFS-7240:
------------------------------------

Thanks for taking notes, and thank you for the lively discussion.  A lot of valid concerns on all sides here. A couple of minor corrections:
{quote}AaronF: Ozone is a lot of new code and Hadoop already has so much code.. {quote}
My concerns particularly are around things that affect stability and operability. Not concerned about lines of code, more about "how manageable" is the codebase in terms of getting a stable release, and supporting in production.

{quote}
shallow data copy is practical only if within same project and daemon otherwise have deal with security setting and coordinations across daemons.
{quote}
We can factor common code, if any, to a shared dependency. I don't see how which repository code lives in really affects fast copy between storage systems.  I can think of ways to do it both within a JVM process consisting of code from multiple git repositories, and via IPC (hand off ownership of a file to another process--not even talking about fancy stuff like shmem).

{quote}
The opponents will raise the same issue as today: show feature parity 
{quote}

I get your concern, but I didn't hear anyone say feature parity. I only heard "integrate with HDFS".  Even integrated with HDFS, there is still a high bar of "utility" to pass, IMO, to justify a very large patch which affects production code.

We all want stable, scalable HDFS. Nobody opposes that ideal. 

I'm not sure trying to evolve HDFS to scale is a better approach than being separate with maybe some shared, well-factored dependencies.  The latter, IMO, could result in better code and dramatically less risk to HDFS.

Appreciate all your hard work thus far and appreciate the challenges you guys face here. I hope you can understand my perspective as well.

> Object store in HDFS
> --------------------
>
>                 Key: HDFS-7240
>                 URL: https://issues.apache.org/jira/browse/HDFS-7240
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Jitendra Nath Pandey
>            Assignee: Jitendra Nath Pandey
>         Attachments: HDFS Scalability and Ozone.pdf, HDFS-7240.001.patch, HDFS-7240.002.patch, HDFS-7240.003.patch, HDFS-7240.003.patch, HDFS-7240.004.patch, HDFS-7240.005.patch, HDFS-7240.006.patch, MeetingMinutes.pdf, Ozone-architecture-v1.pdf, Ozonedesignupdate.pdf, ozone_user_v0.pdf
>
>
> This jira proposes to add object store capabilities into HDFS. 
> As part of the federation work (HDFS-1052) we separated block storage as a generic storage layer. Using the Block Pool abstraction, new kinds of namespaces can be built on top of the storage layer i.e. datanodes.
> In this jira I will explore building an object store using the datanode storage, but independent of namespace metadata.
> I will soon update with a detailed design document.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org