hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dinesh S. Atreya (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9607) Advance Hadoop Architecture (AHA) - HDFS
Date Thu, 07 Jan 2016 00:06:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15086539#comment-15086539

Dinesh S. Atreya commented on HDFS-9607:

Copying [comment | https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15083784&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15083784]
from parent/umbrella JIRA to here:

[Haohui Mai | https://issues.apache.org/jira/secure/ViewProfile.jspa?name=wheat9] added a
comment - Yesterday

I agree that the capabilities can be quite powerful. The real issue how it can be done. There
are some questions need to be answered:

(1) What is the semantic of update-in-place precisely when there are failures? Is it atomic
and transactional? What does the consistent model look like? What are the semantics and durability
guarantee look like? For example, what happens if one of the DN in the pipeline is down? What
will the reader see?
(2) Once you define the semantic, is the semantic / specification meaningful and complete?
Does it cover all the failure cases? How to evaluate and prove there is no corner cases?
(3) How to implement the semantic in code? What is the approach you are taking? Is it MVCC,
distributed transaction or an ad-hoc solution tailored to HDFS?

So far we all agree that it is a useful capability. I don't think it require more communications
to establish it enables a number new use cases.

However, I don't see this is a complete solution without addressing Steve's questions and
all the questions above. It would be beneficial to have a design doc and a working prototype
to clarify the confusions.


> Advance Hadoop Architecture (AHA) - HDFS
> ----------------------------------------
>                 Key: HDFS-9607
>                 URL: https://issues.apache.org/jira/browse/HDFS-9607
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Dinesh S. Atreya
> Link to Umbrella JIRA
> https://issues.apache.org/jira/browse/HADOOP-12620 
> Provide capability to carry out in-place writes/updates. Only writes in-place are supported
where the existing length does not change.
> For example, "Hello World" can be replaced by "Hello HDFS!"
> See https://issues.apache.org/jira/browse/HADOOP-12620?focusedCommentId=15046300&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15046300
for more details.

This message was sent by Atlassian JIRA

View raw message