Return-Path: X-Original-To: apmail-giraph-dev-archive@www.apache.org Delivered-To: apmail-giraph-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9CCEC17BEB for ; Thu, 5 Feb 2015 19:08:38 +0000 (UTC) Received: (qmail 56748 invoked by uid 500); 5 Feb 2015 19:08:38 -0000 Delivered-To: apmail-giraph-dev-archive@giraph.apache.org Received: (qmail 56652 invoked by uid 500); 5 Feb 2015 19:08:38 -0000 Mailing-List: contact dev-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@giraph.apache.org Delivered-To: mailing list dev@giraph.apache.org Received: (qmail 56376 invoked by uid 500); 5 Feb 2015 19:08:38 -0000 Delivered-To: apmail-incubator-giraph-dev@incubator.apache.org Received: (qmail 56373 invoked by uid 99); 5 Feb 2015 19:08:38 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 05 Feb 2015 19:08:38 +0000 Date: Thu, 5 Feb 2015 19:08:38 +0000 (UTC) From: "Hudson (JIRA)" To: giraph-dev@incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (GIRAPH-992) Zookeeper logs have too many NodeExists exceptions MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/GIRAPH-992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14307785#comment-14307785 ] Hudson commented on GIRAPH-992: ------------------------------- ABORTED: Integrated in Giraph-trunk-Commit #1525 (See [https://builds.apache.org/job/Giraph-trunk-Commit/1525/]) GIRAPH-992 Zookeeper logs have too many NodeExists (edunov: http://git-wip-us.apache.org/repos/asf?p=giraph.git&a=commit&h=226fa4bb47a97ac825e05274cba77914f9c96252) * giraph-core/src/main/java/org/apache/giraph/master/BspServiceMaster.java * CHANGELOG * giraph-core/src/main/java/org/apache/giraph/zk/ZooKeeperExt.java > Zookeeper logs have too many NodeExists exceptions > -------------------------------------------------- > > Key: GIRAPH-992 > URL: https://issues.apache.org/jira/browse/GIRAPH-992 > Project: Giraph > Issue Type: Bug > Reporter: Sergey Edunov > Assignee: Sergey Edunov > > There are several places in our code where we do not check properly if zk node already exists before attempting to create it. As a result ZK logs are full of these exceptions. > Biggest offender is recursive path creation in ZooKeeperExt.createExt(). Obviously if part of the path already exists we don't need to create it. > Second biggest offender is writing input splits from master. Here we lunch multiple threads each of them attempting to create the same path. > INFO 2015-02-04 14:32:39,730 [ProcessThread(sid:0 cport:-1):] org.apache.zookeeper.server.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x14b56b9176f0001 type:create cxid:0x1 zxid:0x19 txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201411061513.83344_0001/_masterJobState Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201411061513.83344_0001/_masterJobState > INFO 2015-02-04 14:32:39,740 [ProcessThread(sid:0 cport:-1):] org.apache.zookeeper.server.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x14b56b9176f0001 type:create cxid:0x3 zxid:0x1a txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201411061513.83344_0001/_applicationAttemptsDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201411061513.83344_0001/_applicationAttemptsDir > INFO 2015-02-04 14:32:39,742 [ProcessThread(sid:0 cport:-1):] org.apache.zookeeper.server.PrepRequestProcessor - Got user-level KeeperException when processing sessionid:0x14b56b9176f0001 type:create cxid:0x5 zxid:0x1b txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201411061513.83344_0001/_applicationAttemptsDir/0/_superstepDir Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201411061513.83344_0001/_applicationAttemptsDir/0/_superstepDir -- This message was sent by Atlassian JIRA (v6.3.4#6332)