spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (SPARK-20884) Spark' masters will be both standby due to the bug of curator
Date Tue, 06 Jun 2017 09:31:18 GMT

     [ https://issues.apache.org/jira/browse/SPARK-20884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sean Owen resolved SPARK-20884.
-------------------------------
    Resolution: Won't Fix

We can't merge this change, because Curator 2.8 won't work with Curator 2.6 environments,
which include Hadoop 2.6/2.7 envs, which are still supported. Much later we could update this,
basically when Spark 3 moves on to Hadoop 3 (which uses Curator 2.12)

> Spark' masters will be both standby due to the bug of curator 
> --------------------------------------------------------------
>
>                 Key: SPARK-20884
>                 URL: https://issues.apache.org/jira/browse/SPARK-20884
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 2.1.1
>            Reporter: liuzhaokun
>
> I built a cluster with two masters and three workers.When there is a switch of master's
state,two masters are both standby in a long period.There are some ERROR in master's logfile
" ERROR CuratorFrameworkImpl: Background exception was not retry-able or retry gave up
> java.lang.IllegalArgumentException: Path must start with / character
> 	at org.apache.curator.utils.PathUtils.validatePath(PathUtils.java:53)
> 	at org.apache.curator.utils.ZKPaths.getNodeFromPath(ZKPaths.java:56)
> 	at org.apache.curator.framework.recipes.leader.LeaderLatch.checkLeadership(LeaderLatch.java:421)
> 	at org.apache.curator.framework.recipes.leader.LeaderLatch.access$500(LeaderLatch.java:60)
> 	at org.apache.curator.framework.recipes.leader.LeaderLatch$6.processResult(LeaderLatch.java:478)
> 	at org.apache.curator.framework.imps.CuratorFrameworkImpl.sendToBackgroundCallback(CuratorFrameworkImpl.java:686)
> 	at org.apache.curator.framework.imps.CuratorFrameworkImpl.processBackgroundOperation(CuratorFrameworkImpl.java:485)
> 	at org.apache.curator.framework.imps.GetChildrenBuilderImpl$2.processResult(GetChildrenBuilderImpl.java:166)
> 	at org.apache.zookeeper.ClientCnxn$EventThread.processEvent(ClientCnxn.java:587)
> 	at org.apache.zookeeper.ClientCnxn$EventThread.run(ClientCnxn.java:495)"
> it can be related to https://issues.apache.org/jira/browse/CURATOR-168.
> So I think the version of curator in spark should be 2.8.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message