hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1929) DeadLock in RM when automatic failover is enabled.
Date Fri, 11 Apr 2014 18:38:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-1929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13966941#comment-13966941
] 

Karthik Kambatla commented on YARN-1929:
----------------------------------------

Looks a little tricky. Observations from reading the code again:
# EmbeddedElectorService(EES) doesn't need to synchronize the serviceStart and serviceStop
methods. These methods call synchronized methods in ActiveStandbyElector(ASE). Actually, given
that the ActiveStandbyElectorCallback methods implemented by EES are also called from ASE
in-process, we probably don't need to synchronize them either.
# AdminService(AS) doesn't need to synchronize the service* methods either. AS#{start|stop}Server
can be synchronized if need be.
# Removing above synchronizations alone wouldn't be enough because there would still be a
deadlock between (*ASE#processResult* -> EES#becomeActive -> *AS#transitionToActive*
-> *RM#transitionToActive*) and (RM#serviceStop -> RM.super#serviceStop -> *RM.super#stop*
-> AS#stop -> AS#serviceStop -> EES#serviceStop -> *ASE#quitElection*), the methods
in bold are synchronized.

We should probably revisit the synchronization of CompositeService#stop. [~steve_l@iseran.com]
- any suggestions?

> DeadLock in RM when automatic failover is enabled.
> --------------------------------------------------
>
>                 Key: YARN-1929
>                 URL: https://issues.apache.org/jira/browse/YARN-1929
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>         Environment: Yarn HA cluster
>            Reporter: Rohith
>            Assignee: Karthik Kambatla
>            Priority: Blocker
>
> Dead lock detected  in RM when automatic failover is enabled.
> {noformat}
> Found one Java-level deadlock:
> =============================
> "Thread-2":
>   waiting to lock monitor 0x00007fb514303cf0 (object 0x00000000ef153fd0, a org.apache.hadoop.ha.ActiveStandbyElector),
>   which is held by "main-EventThread"
> "main-EventThread":
>   waiting to lock monitor 0x00007fb514750a48 (object 0x00000000ef154020, a org.apache.hadoop.yarn.server.resourcemanager.EmbeddedElectorService),
>   which is held by "Thread-2"
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message