hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5694) ZKRMStateStore should always start its verification thread to prevent accidental state store corruption
Date Thu, 10 Nov 2016 18:01:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-5694?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15654697#comment-15654697

Jian He commented on YARN-5694:

bq. If we agree that it's bad to have two RMs accidentally sharing the same state store, 
If it's in non-HA mode, currently there's no protection in the ZKStore preventing two RMs
from sharing the same store. All the ACLs setting related code is only used in HA mode. Essentially,
with current patch, I doubt it will get NoAuthException in the verifyThread, without making
user change the ACLs manually. So the handling code in this patch will not be triggered with
default setting. Maybe I'm wrong, you may try on a real cluster..

bq. why would you not want to catch the issue as early as possible?
My point is that first,will this code work as mentioned above. second, if there's no difference
in terms of functionality, why do I need to start a thread pinging the zk continuously every
few seconds.  Of course, I might miss something, you may clarify more...

Also, is the use-case mainly about two clusters sharing the same zk-store with the same path
?  IMHO, this is not a primary use-case to solve, if user mis-configured, it's user's fault.
There are many other places that can go wrong.  e.g. if two clusters configure the same path
for anything on HDFS.

If the use-case is about two RMs sharing the same zk-path in the same cluster with non-HA
mode. I think in non-HA mode, the invalid RM will not take workload in the first place, clients,
NMs will not switch to that RM if HA is not configured properly. 

> ZKRMStateStore should always start its verification thread to prevent accidental state
store corruption
> -------------------------------------------------------------------------------------------------------
>                 Key: YARN-5694
>                 URL: https://issues.apache.org/jira/browse/YARN-5694
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 3.0.0-alpha1
>            Reporter: Daniel Templeton
>            Assignee: Daniel Templeton
>            Priority: Critical
>              Labels: oct16-medium
>         Attachments: YARN-5694.001.patch, YARN-5694.002.patch, YARN-5694.003.patch, YARN-5694.004.patch,
YARN-5694.004.patch, YARN-5694.005.patch, YARN-5694.006.patch, YARN-5694.007.patch, YARN-5694.branch-2.7.001.patch,
> There are two cases.  In branch-2.7, the {{ZKRMStateStore.VerifyActiveStatusThread}}
is always started, even when using embedded or Curator failover.  In branch-2.8, the {{ZKRMStateStore.VerifyActiveStatusThread}}
is only started when HA is disabled, which makes no sense.  Based on the JIRA that introduced
that change (YARN-4559), I believe the intent was to start it only when embedded failover
is disabled.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message