manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wright (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CONNECTORS-1036) Agents shutdown can cause Zookeeper error
Date Wed, 17 Sep 2014 14:43:34 GMT

    [ https://issues.apache.org/jira/browse/CONNECTORS-1036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14137339#comment-14137339
] 

Karl Wright commented on CONNECTORS-1036:
-----------------------------------------

Unfortunately, this issue requires a structural change to ZooKeeperLockManager in order to
be fixed.  Essentially, ZooKeeperConnection objects have to have a sticky association with
non-lock ephemeral nodes.  (Lock ephemeral nodes already have this sticky association.)  That
means:

-- We need to keep track of the zookeeper connection associated with each non-lock ephemeral
node
-- There needs to be a table keyed by nodepath somewhere, which points to an object
    which caches the ZookeeperConnection object
    It could even be the connection object itself.
-- When working with a given path, the ZookeeperLockManager code has to be careful to look
up
    the appropriate connection for the ephemeral path
-- Each ZookeeperConnection object is responsible for maintaining at most ONE non-lock ephemeral
node,
    so its API methods can skip specification of the nodepath whenever writing is taking place

It may be best to introduce a ZooKeeperEphemeralNode class that manages connection affinity,
and perhaps even a ZooKeeperEphemeralNodeManager static class to manage the set of outstanding
ZooKeeperEphemeralNode objects.




> Agents shutdown can cause Zookeeper error
> -----------------------------------------
>
>                 Key: CONNECTORS-1036
>                 URL: https://issues.apache.org/jira/browse/CONNECTORS-1036
>             Project: ManifoldCF
>          Issue Type: Bug
>          Components: Framework core
>    Affects Versions: Manifold 1.7.1, ManifoldCF 2.0
>            Reporter: Karl Wright
>            Assignee: Karl Wright
>             Fix For: Manifold 1.7.1, ManifoldCF 2.0
>
>
> Here's the exception:
> {code}
> WARN 2014-09-17 14:06:52,228 (Shutdown thread) - Exception tossed on repository connector
pool cleanup: KeeperErrorCode = NoNode for /org.apache.manifoldcf.serviceactive-_REPOSITORYCONNECTORPOOL_Web-_ANON_5
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: KeeperErrorCode = NoNode for
/org.apache.manifoldcf.serviceactive-_REPOSITORYCONNECTORPOOL_Web-_ANON_5
>         at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.handleKeeperException(ZooKeeperConnection.java:941)
>         at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.deleteNode(ZooKeeperConnection.java:155)
>         at org.apache.manifoldcf.core.lockmanager.ZooKeeperLockManager.endServiceActivity(ZooKeeperLockManager.java:478)
>         at org.apache.manifoldcf.core.connectorpool.ConnectorPool$Pool.releaseAll(ConnectorPool.java:735)
>         at org.apache.manifoldcf.core.connectorpool.ConnectorPool.closeAllConnectors(ConnectorPool.java:381)
>         at org.apache.manifoldcf.crawler.repositoryconnectorpool.RepositoryConnectorPool.closeAllConnectors(RepositoryConnectorPool.java:144)
>         at org.apache.manifoldcf.crawler.system.ManifoldCF.localCleanup(ManifoldCF.java:110)
>         at org.apache.manifoldcf.crawler.system.CrawlerAgent.cleanUp(CrawlerAgent.java:105)
>         at org.apache.manifoldcf.agents.system.AgentsDaemon.stopAgents(AgentsDaemon.java:171)
>         at org.apache.manifoldcf.agents.system.AgentsDaemon$AgentsShutdownHook.doCleanup(AgentsDaemon.java:386)
>         at org.apache.manifoldcf.core.system.ManifoldCF.cleanUpEnvironment(ManifoldCF.java:1295)
>         at org.apache.manifoldcf.core.system.ManifoldCF$ShutdownThread.run(ManifoldCF.java:1483)
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode
for /org.apache.manifoldcf.serviceactive-_REPOSITORYCONNECTORPOOL_Web-_ANON_5
>         at org.apache.zookeeper.KeeperException.create(KeeperException.java:111)
>         at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
>         at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873)
>         at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.deleteNode(ZooKeeperConnection.java:150)
>         ... 10 more
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message