hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ayush Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-14316) RBF: Support unavailable subclusters for mount points with multiple destinations
Date Wed, 27 Mar 2019 11:18:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-14316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16802675#comment-16802675

Ayush Saxena commented on HDFS-14316:

That I agree the test covers write in case of fault.  I am not telling that won't work. My
concern is after that works, what inconsistencies will it lead to. We can't be like on account
of Fault tolerance we can let inconsistencies pitch in after. And if it was a normal case
or a case that we expect to occur rarely, That shall be still OK. But here these inconsistencies
shall come up when the actual directory where it was supposed to write was not available,
means when the intended cluster wasn't  available, for which we intend to tolerate the
fault in idle case and if this is a regular scenario. I don't think we should get that on
ground and put the risk on Admin head. This will come back to us here only with a broken by


Anyway we shall be providing a mechanism for fault tolerance just not with two ORDERS HASH
and LOCAL(with local now write might be not local too. I guess semantics for this also shall
be compromised).So this feature shall be usable. 


I am not blindly against doing it. But just want to prevent not breaking the further cases,
and  I don't want to put  this to ADMIN at this level where chances being safe aren't
that high.


Do you have a use case losing if we don't support on HASH and LOCAL. Otherwise we may play
safe and let it go with 3 orders as of now and might try with other two Orders separately
and see there how much inconsistency we can handle and may get more opinions there. :)


> RBF: Support unavailable subclusters for mount points with multiple destinations
> --------------------------------------------------------------------------------
>                 Key: HDFS-14316
>                 URL: https://issues.apache.org/jira/browse/HDFS-14316
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Íñigo Goiri
>            Assignee: Íñigo Goiri
>            Priority: Major
>         Attachments: HDFS-14316-HDFS-13891.000.patch, HDFS-14316-HDFS-13891.001.patch,
HDFS-14316-HDFS-13891.002.patch, HDFS-14316-HDFS-13891.003.patch, HDFS-14316-HDFS-13891.004.patch,
HDFS-14316-HDFS-13891.005.patch, HDFS-14316-HDFS-13891.006.patch, HDFS-14316-HDFS-13891.007.patch,
HDFS-14316-HDFS-13891.008.patch, HDFS-14316-HDFS-13891.009.patch, HDFS-14316-HDFS-13891.010.patch,
HDFS-14316-HDFS-13891.011.patch, HDFS-14316-HDFS-13891.012.patch, HDFS-14316-HDFS-13891.013.patch
> Currently mount points with multiple destinations (e.g., HASH_ALL) fail writes when the
destination subcluster is down. We need an option to allow writing in other subclusters when
one is down.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message