manifoldcf-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karl Wright (JIRA)" <>
Subject [jira] [Commented] (CONNECTORS-1036) Agents shutdown can cause Zookeeper error
Date Wed, 17 Sep 2014 14:43:34 GMT


Karl Wright commented on CONNECTORS-1036:

Unfortunately, this issue requires a structural change to ZooKeeperLockManager in order to
be fixed.  Essentially, ZooKeeperConnection objects have to have a sticky association with
non-lock ephemeral nodes.  (Lock ephemeral nodes already have this sticky association.)  That

-- We need to keep track of the zookeeper connection associated with each non-lock ephemeral
-- There needs to be a table keyed by nodepath somewhere, which points to an object
    which caches the ZookeeperConnection object
    It could even be the connection object itself.
-- When working with a given path, the ZookeeperLockManager code has to be careful to look
    the appropriate connection for the ephemeral path
-- Each ZookeeperConnection object is responsible for maintaining at most ONE non-lock ephemeral
    so its API methods can skip specification of the nodepath whenever writing is taking place

It may be best to introduce a ZooKeeperEphemeralNode class that manages connection affinity,
and perhaps even a ZooKeeperEphemeralNodeManager static class to manage the set of outstanding
ZooKeeperEphemeralNode objects.

> Agents shutdown can cause Zookeeper error
> -----------------------------------------
>                 Key: CONNECTORS-1036
>                 URL:
>             Project: ManifoldCF
>          Issue Type: Bug
>          Components: Framework core
>    Affects Versions: Manifold 1.7.1, ManifoldCF 2.0
>            Reporter: Karl Wright
>            Assignee: Karl Wright
>             Fix For: Manifold 1.7.1, ManifoldCF 2.0
> Here's the exception:
> {code}
> WARN 2014-09-17 14:06:52,228 (Shutdown thread) - Exception tossed on repository connector
pool cleanup: KeeperErrorCode = NoNode for /org.apache.manifoldcf.serviceactive-_REPOSITORYCONNECTORPOOL_Web-_ANON_5
> org.apache.manifoldcf.core.interfaces.ManifoldCFException: KeeperErrorCode = NoNode for
>         at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.handleKeeperException(
>         at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.deleteNode(
>         at org.apache.manifoldcf.core.lockmanager.ZooKeeperLockManager.endServiceActivity(
>         at org.apache.manifoldcf.core.connectorpool.ConnectorPool$Pool.releaseAll(
>         at org.apache.manifoldcf.core.connectorpool.ConnectorPool.closeAllConnectors(
>         at org.apache.manifoldcf.crawler.repositoryconnectorpool.RepositoryConnectorPool.closeAllConnectors(
>         at org.apache.manifoldcf.crawler.system.ManifoldCF.localCleanup(
>         at org.apache.manifoldcf.crawler.system.CrawlerAgent.cleanUp(
>         at org.apache.manifoldcf.agents.system.AgentsDaemon.stopAgents(
>         at org.apache.manifoldcf.agents.system.AgentsDaemon$AgentsShutdownHook.doCleanup(
>         at org.apache.manifoldcf.core.system.ManifoldCF.cleanUpEnvironment(
>         at org.apache.manifoldcf.core.system.ManifoldCF$
> Caused by: org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode = NoNode
for /org.apache.manifoldcf.serviceactive-_REPOSITORYCONNECTORPOOL_Web-_ANON_5
>         at org.apache.zookeeper.KeeperException.create(
>         at org.apache.zookeeper.KeeperException.create(
>         at org.apache.zookeeper.ZooKeeper.delete(
>         at org.apache.manifoldcf.core.lockmanager.ZooKeeperConnection.deleteNode(
>         ... 10 more
> {code}

This message was sent by Atlassian JIRA

View raw message