giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Edunov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (GIRAPH-992) Zookeeper logs have too many NodeExists exceptions
Date Thu, 05 Feb 2015 00:35:35 GMT
Sergey Edunov created GIRAPH-992:
------------------------------------

             Summary: Zookeeper logs have too many NodeExists exceptions
                 Key: GIRAPH-992
                 URL: https://issues.apache.org/jira/browse/GIRAPH-992
             Project: Giraph
          Issue Type: Bug
            Reporter: Sergey Edunov


There are several places in our code where we do not check properly if zk node already exists
before attempting to create it. As a result ZK logs are full of these exceptions. 
Biggest offender is recursive path creation in ZooKeeperExt.createExt(). Obviously if part
of the path already exists we don't need to create it. 
Second biggest offender is writing input splits from master. Here we lunch multiple threads
each of them attempting to create the same path. 

INFO    2015-02-04 14:32:39,730 [ProcessThread(sid:0 cport:-1):] org.apache.zookeeper.server.PrepRequestProcessor
 - Got user-level KeeperException when processing sessionid:0x14b56b9176f0001 type:create
cxid:0x1 zxid:0x19 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201411061513.83344_0001/_masterJobState
Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201411061513.83344_0001/_masterJobState
INFO    2015-02-04 14:32:39,740 [ProcessThread(sid:0 cport:-1):] org.apache.zookeeper.server.PrepRequestProcessor
 - Got user-level KeeperException when processing sessionid:0x14b56b9176f0001 type:create
cxid:0x3 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201411061513.83344_0001/_applicationAttemptsDir
Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201411061513.83344_0001/_applicationAttemptsDir
INFO    2015-02-04 14:32:39,742 [ProcessThread(sid:0 cport:-1):] org.apache.zookeeper.server.PrepRequestProcessor
 - Got user-level KeeperException when processing sessionid:0x14b56b9176f0001 type:create
cxid:0x5 zxid:0x1b txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201411061513.83344_0001/_applicationAttemptsDir/0/_superstepDir
Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201411061513.83344_0001/_applicationAttemptsDir/0/_superstepDir




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message