Return-Path: X-Original-To: apmail-giraph-user-archive@www.apache.org Delivered-To: apmail-giraph-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 76D5B10EDB for ; Wed, 30 Oct 2013 19:49:44 +0000 (UTC) Received: (qmail 1478 invoked by uid 500); 30 Oct 2013 19:49:42 -0000 Delivered-To: apmail-giraph-user-archive@giraph.apache.org Received: (qmail 1008 invoked by uid 500); 30 Oct 2013 19:49:35 -0000 Mailing-List: contact user-help@giraph.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@giraph.apache.org Delivered-To: mailing list user@giraph.apache.org Received: (qmail 865 invoked by uid 99); 30 Oct 2013 19:49:30 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Oct 2013 19:49:30 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,WEIRD_PORT X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of artie.pesh-imam@tapad.com designates 209.85.128.52 as permitted sender) Received: from [209.85.128.52] (HELO mail-qe0-f52.google.com) (209.85.128.52) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Oct 2013 19:49:21 +0000 Received: by mail-qe0-f52.google.com with SMTP id w7so1130529qeb.25 for ; Wed, 30 Oct 2013 12:48:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:date:message-id:subject:from:to :content-type; bh=Xl45TT4rGTYX/ZS2O6HYnmh7hgaY6dpoeR6KYhD+iGg=; b=grQf8oPIQfnrSpmiBRl+98XE3NidhsEcyJDmKHrEY216xa0v8psVRgHEDqDok12F6y mRfDPu9PIo9xJo90ePdzrj0bzXxrFl8UWLt0FJRp7s65yVL6tw6KJ+jrpjtXk1b6vl73 EdaU5k7bo7D3Q47RXKvYsejZGj4svKSHGMwMw6YsZ1r7E/AphHCwa8QOn/uIEjdfiEQ5 00HWxFRP2PcU3nw5wzqaFUh+nnJPmyS+398t8P3+Z0sriM78n2TJsxeCzGhXrDD3bS2G 0+gWlYqjq4SXXc9RLYs661zWXc78TagGBxecQCjQABPgMewO1t7MiZ9Qd2xvYhWLvFAY 5XlA== X-Gm-Message-State: ALoCoQm/xfXim5y+PaLIqz4ke6tBmv+gJqR8px8+1y06d183R/gViiX7ho5KiR9SbiQwA4obQJFl MIME-Version: 1.0 X-Received: by 10.49.110.36 with SMTP id hx4mr5069587qeb.93.1383162539304; Wed, 30 Oct 2013 12:48:59 -0700 (PDT) Received: by 10.96.55.100 with HTTP; Wed, 30 Oct 2013 12:48:59 -0700 (PDT) Date: Wed, 30 Oct 2013 15:48:59 -0400 Message-ID: Subject: Issue Running Giraph Job From: Artie Pesh-Imam To: user@giraph.apache.org Content-Type: multipart/alternative; boundary=047d7bdc94ccd2aaa404e9fa9fc1 X-Virus-Checked: Checked by ClamAV on apache.org --047d7bdc94ccd2aaa404e9fa9fc1 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi all, I'm able to run this job locally but when I try to run against our cluster, the job seems to fail. Here's the log entries. It looks like it's not proceeding beyond superstep 0. Is there anyway to get a more detailed picture of what's going on? 2013-10-30 19:38:04,467 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead 2013-10-30 19:38:05,157 WARN org.apache.hadoop.conf.Configuration: session.id is deprecated. Instead, use dfs.metrics.session-id 2013-10-30 19:38:05,158 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics with processName=3DMAP, sessionId=3D 2013-10-30 19:38:05,634 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit code 0 2013-10-30 19:38:05,639 INFO org.apache.hadoop.mapred.Task: Using ResourceCalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@371e88fb 2013-10-30 19:38:06,029 INFO org.apache.hadoop.mapred.MapTask: Processing split: 'org.apache.giraph.bsp.BspInputSplit, index=3D-1, num=3D-1 2013-10-30 19:38:06,047 INFO org.apache.giraph.graph.GraphTaskManager: setup: Log level remains at info 2013-10-30 19:38:06,117 INFO org.apache.giraph.graph.GraphTaskManager: Distributed cache is empty. Assuming fatjar. 2013-10-30 19:38:06,117 INFO org.apache.giraph.graph.GraphTaskManager: setup: classpath @ /d02/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jobcache/job_2013103015= 59_0024/jars/job.jar for job jobs.ConnectedComponents 2013-10-30 19:38:06,155 INFO org.apache.giraph.zk.ZooKeeperManager: createCandidateStamp: Made the directory _bsp/_defaultZkManagerDir/job_201310301559_0024 2013-10-30 19:38:06,166 INFO org.apache.giraph.zk.ZooKeeperManager: createCandidateStamp: Creating my filestamp _bsp/_defaultZkManagerDir/job_201310301559_0024/_task/ datanode05.prd.nj1.tapad.com 0 2013-10-30 19:38:06,216 INFO org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList: Got [ datanode05.prd.nj1.tapad.com] 1 hosts from 1 candidates when 1 required (polling period is 3000) on attempt 0 2013-10-30 19:38:06,217 INFO org.apache.giraph.zk.ZooKeeperManager: createZooKeeperServerList: Creating the final ZooKeeper file '_bsp/_defaultZkManagerDir/job_201310301559_0024/ zkServerList_datanode05.prd.nj1.tapad.com 0 ' 2013-10-30 19:38:06,235 INFO org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList: For task 0, got file 'zkServerList_datanode05.prd.nj1.tapad.com 0 ' (polling period is 3000) 2013-10-30 19:38:06,235 INFO org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList: Found [datanode05.prd.nj1.tapad.com, 0] 2 hosts in filename 'zkServerList_datanode05.prd.nj1.tapad.com 0 ' 2013-10-30 19:38:06,236 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Trying to delete old directory /d06/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jobcache/job_2013103015= 59_0024/work/_bspZooKeeper 2013-10-30 19:38:06,243 INFO org.apache.giraph.zk.ZooKeeperManager: generateZooKeeperConfigFile: Creating file /d06/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jobcache/job_2013103015= 59_0024/work/_bspZooKeeper/zoo.cfg in /d06/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jobcache/job_2013103015= 59_0024/work/_bspZooKeeper with base port 22181 2013-10-30 19:38:06,243 INFO org.apache.giraph.zk.ZooKeeperManager: generateZooKeeperConfigFile: Make directory of _bspZooKeeper =3D true 2013-10-30 19:38:06,243 INFO org.apache.giraph.zk.ZooKeeperManager: generateZooKeeperConfigFile: Delete of zoo.cfg =3D false 2013-10-30 19:38:06,246 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Attempting to start ZooKeeper server with command [/usr/java/jdk1.7.0_40/jre/bin/java, -Xmx512m, -XX:ParallelGCThreads=3D4, -XX:+UseConcMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=3D70, -XX:MaxGCPauseMillis=3D100, -cp, /d02/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jobcache/job_2013103015= 59_0024/jars/job.jar, org.apache.zookeeper.server.quorum.QuorumPeerMain, /d06/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jobcache/job_2013103015= 59_0024/work/_bspZooKeeper/zoo.cfg] in directory /d06/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jobcache/job_2013103015= 59_0024/work/_bspZooKeeper 2013-10-30 19:38:06,249 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Shutdown hook added. 2013-10-30 19:38:06,249 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Connect attempt 0 of 10 max trying to connect to datanode05.prd.nj1.tapad.com:22181with poll msecs =3D 3000 2013-10-30 19:38:06,253 WARN org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Got ConnectException java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339= ) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.j= ava:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManag= er.java:701) at org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskMan= ager.java:357) at org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:188) at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:60) at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:90) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.j= ava:1408) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2013-10-30 19:38:09,256 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Connect attempt 1 of 10 max trying to connect to datanode05.prd.nj1.tapad.com:22181 with poll msecs =3D 3000 2013-10-30 19:38:09,256 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Connected to datanode05.prd.nj1.tapad.com/10.20.20.31:22181! 2013-10-30 19:38:09,256 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers: Creating my filestamp _bsp/_defaultZkManagerDir/job_201310301559_0024/_zkServer/ datanode05.prd.nj1.tapad.com 0 2013-10-30 19:38:09,291 INFO org.apache.giraph.graph.GraphTaskManager: setup: Chosen to run ZooKeeper... 2013-10-30 19:38:09,291 INFO org.apache.giraph.graph.GraphTaskManager: setup: Starting up BspServiceMaster (master thread)... 2013-10-30 19:38:09,306 INFO org.apache.giraph.bsp.BspService: BspService: Connecting to ZooKeeper with job job_201310301559_0024, 0 on datanode05.prd.nj1.tapad.com:22181 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3D3.4.5-cdh4.4.0--1, built on 09/03/2013 16:1= 4 GMT 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=3Ddatanode05.prd.nj1.tapad.com 2013-10-30 19:38:09,31= 7 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=3D1.7.0_40 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=3DOracle Corporation 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=3D/usr/java/jdk1.7.0_40/jre 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=3D/var/run/cloudera-scm-agent/process/238-mapre= duce-TASKTRACKER:/usr/java/jdk1.7.0_40/lib/tools.jar:/opt/cloudera/parcels/= CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce:/opt/cloudera/parcels/= CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-= cdh4.4.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.= 20-mapreduce/lib/activation-1.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.= 4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/ant-contrib-1.0b3.jar:/opt/cloudera= /parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/asm-3.2.j= ar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapredu= ce/lib/avro-1.7.4.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/= hadoop-0.20-mapreduce/lib/avro-compiler-1.7.4.jar:/opt/cloudera/parcels/CDH= -4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-1.7= .0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-map= reduce/lib/commons-beanutils-core-1.8.0.jar:/opt/cloudera/parcels/CDH-4.4.0= -1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/commons-cli-1.2.jar:/opt/cl= oudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/com= mons-codec-1.4.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/had= oop-0.20-mapreduce/lib/commons-collections-3.2.1.jar:/opt/cloudera/parcels/= CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/commons-compress-1= .4.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-m= apreduce/lib/commons-configuration-1.6.jar:/opt/cloudera/parcels/CDH-4.4.0-= 1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/commons-digester-1.8.jar:/op= t/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib= /commons-el-1.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/ha= doop-0.20-mapreduce/lib/commons-httpclient-3.1.jar:/opt/cloudera/parcels/CD= H-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/commons-io-2.1.jar:/= opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/l= ib/commons-lang-2.5.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/li= b/hadoop-0.20-mapreduce/lib/commons-logging-1.1.1.jar:/opt/cloudera/parcels= /CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/commons-math-2.1.= jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapred= uce/lib/commons-net-3.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.3= 9/lib/hadoop-0.20-mapreduce/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-= 4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/hadoop-fairscheduler-2= .0.0-mr1-cdh4.4.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/= hadoop-0.20-mapreduce/lib/hsqldb-1.8.0.10.jar:/opt/cloudera/parcels/CDH-4.4= .0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jackson-core-asl-1.8.8.ja= r:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduc= e/lib/jackson-jaxrs-1.8.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0= .39/lib/hadoop-0.20-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/opt/clouder= a/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jackson-= xc-1.8.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.= 20-mapreduce/lib/jasper-compiler-5.5.23.jar:/opt/cloudera/parcels/CDH-4.4.0= -1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jasper-runtime-5.5.23.jar:/= opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/l= ib/jaxb-api-2.2.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/= hadoop-0.20-mapreduce/lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-4= .4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jersey-core-1.8.jar:/op= t/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib= /jersey-json-1.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/h= adoop-0.20-mapreduce/lib/jersey-server-1.8.jar:/opt/cloudera/parcels/CDH-4.= 4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jets3t-0.6.1.jar:/opt/cl= oudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jet= tison-1.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0= .20-mapreduce/lib/jetty-6.1.26.cloudera.2.jar:/opt/cloudera/parcels/CDH-4.4= .0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jetty-util-6.1.26.clouder= a.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-ma= preduce/lib/jline-0.9.94.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.= 39/lib/hadoop-0.20-mapreduce/lib/jsch-0.1.42.jar:/opt/cloudera/parcels/CDH-= 4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar:/opt/c= loudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/js= r305-1.3.9.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-= 0.20-mapreduce/lib/junit-4.8.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4= .0.p0.39/lib/hadoop-0.20-mapreduce/lib/kfs-0.2.2.jar:/opt/cloudera/parcels/= CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/kfs-0.3.jar:/opt/c= loudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/lo= g4j-1.2.17.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-= 0.20-mapreduce/lib/mockito-all-1.8.5.jar:/opt/cloudera/parcels/CDH-4.4.0-1.= cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/paranamer-2.3.jar:/opt/clouder= a/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/protobuf= -java-2.4.0a.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoo= p-0.20-mapreduce/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-4.4.0-1.= cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/slf4j-api-1.6.1.jar:/opt/cloud= era/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/snappy= -java-1.0.4.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hado= op-0.20-mapreduce/lib/stax-api-1.0.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.= cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/xmlenc-0.52.jar:/opt/cloudera/= parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/xz-1.0.jar= :/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce= /lib/zookeeper-3.4.5-cdh4.4.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.= 0.p0.39/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-2.1.jar:/opt/cloudera/par= cels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-a= pi-2.1.jar:/usr/share/cmf/lib/plugins/tt-instrumentation-4.7.2.jar:/usr/sha= re/cmf/lib/plugins/navigator-plugin-4.7.2-shaded.jar:/usr/share/cmf/lib/plu= gins/event-publish-4.7.2-shaded.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.= 4.0.p0.39/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH= -4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/opt/cloudera/p= arcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23= .jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/j= ersey-core-1.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/had= oop-hdfs/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0= .39/lib/hadoop-hdfs/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-4.4.0-1.= cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/zookeeper-3.4.5-cdh4.4.0.jar:/opt/cloude= ra/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/servlet-api-2.5.j= ar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/com= mons-logging-1.1.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib= /hadoop-hdfs/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.= 39/lib/hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/opt/cloudera/parce= ls/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.= jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/pr= otobuf-java-2.4.0a.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib= /hadoop-hdfs/lib/commons-lang-2.5.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh= 4.4.0.p0.39/lib/hadoop-hdfs/lib/jersey-server-1.8.jar:/opt/cloudera/parcels= /CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/opt/cloude= ra/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/commons-el-1.0.ja= r:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/comm= ons-codec-1.4.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hado= op-hdfs/lib/commons-io-2.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p= 0.39/lib/hadoop-hdfs/lib/jetty-6.1.26.cloudera.2.jar:/opt/cloudera/parcels/= CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/opt/cloude= ra/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/jackson-core-asl-= 1.8.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/= lib/commons-daemon-1.0.3.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.= 39/lib/hadoop-hdfs/lib/jsr305-1.3.9.jar:/opt/cloudera/parcels/CDH-4.4.0-1.c= dh4.4.0.p0.39/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.4.0.jar:/opt/cloudera/= parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.4= .0-tests.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hd= fs/hadoop-hdfs.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/had= oop/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.3= 9/lib/hadoop/lib/jsp-api-2.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0= .p0.39/lib/hadoop/lib/jasper-runtime-5.5.23.jar:/opt/cloudera/parcels/CDH-4= .4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jackson-xc-1.8.8.jar:/opt/cloudera/par= cels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jettison-1.1.jar:/opt/cloude= ra/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jersey-core-1.8.jar:/o= pt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/guava-11.0.2.= jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/log4j-1= .2.17.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/c= ommons-collections-3.2.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.= 39/lib/hadoop/lib/zookeeper-3.4.5-cdh4.4.0.jar:/opt/cloudera/parcels/CDH-4.= 4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/servlet-api-2.5.jar:/opt/cloudera/parce= ls/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/o= pt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/commons-loggi= ng-1.1.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/li= b/asm-3.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/l= ib/jetty-util-6.1.26.cloudera.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.= 4.0.p0.39/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/opt/cloudera/parcels= /CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jsch-0.1.42.jar:/opt/cloudera/pa= rcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/o= pt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/commons-lang-= 2.5.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/com= mons-httpclient-3.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/li= b/hadoop/lib/jersey-server-1.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4= .0.p0.39/lib/hadoop/lib/xmlenc-0.52.jar:/opt/cloudera/parcels/CDH-4.4.0-1.c= dh4.4.0.p0.39/lib/hadoop/lib/commons-digester-1.8.jar:/opt/cloudera/parcels= /CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jaxb-api-2.2.2.jar:/opt/cloudera= /parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:= /opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jersey-json= -1.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/ja= ckson-jaxrs-1.8.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/= hadoop/lib/commons-el-1.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0= .39/lib/hadoop/lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH-4.4= .0-1.cdh4.4.0.p0.39/lib/hadoop/lib/commons-codec-1.4.jar:/opt/cloudera/parc= els/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/mockito-all-1.8.5.jar:/opt/cl= oudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/commons-io-2.1.jar= :/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/slf4j-api-= 1.6.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/j= etty-6.1.26.cloudera.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39= /lib/hadoop/lib/junit-4.8.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.= p0.39/lib/hadoop/lib/paranamer-2.3.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cd= h4.4.0.p0.39/lib/hadoop/lib/jline-0.9.94.jar:/opt/cloudera/parcels/CDH-4.4.= 0-1.cdh4.4.0.p0.39/lib/hadoop/lib/avro-1.7.4.jar:/opt/cloudera/parcels/CDH-= 4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/activation-1.1.jar:/opt/cloudera/parc= els/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/opt/= cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/hue-plugins-2.5.= 0-cdh4.4.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/= lib/kfs-0.3.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop= /lib/commons-math-2.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/= lib/hadoop/lib/jackson-core-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1= .cdh4.4.0.p0.39/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/opt/cloude= ra/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/commons-net-3.1.jar:/o= pt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/stax-api-1.0.= 1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jaxb-= impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoo= p/lib/xz-1.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoo= p/lib/commons-configuration-1.6.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.= 4.0.p0.39/lib/hadoop/lib/jsr305-1.3.9.jar:/opt/cloudera/parcels/CDH-4.4.0-1= .cdh4.4.0.p0.39/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/opt/cloudera/par= cels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jets3t-0.6.1.jar:/opt/cloude= ra/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/hadoop-common-2.0.0-cdh4.4= .0-tests.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/ha= doop-annotations-2.0.0-cdh4.4.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.= 4.0.p0.39/lib/hadoop/hadoop-auth.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4= .4.0.p0.39/lib/hadoop/hadoop-auth-2.0.0-cdh4.4.0.jar:/opt/cloudera/parcels/= CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/hadoop-common-2.0.0-cdh4.4.0.jar:/opt= /cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/hadoop-common.jar:/= opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/hadoop-annotatio= ns.jar:/d02/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jobcache/job_201= 310301559_0024/jars/classes:/d02/dfs/dn/tasktracker/taskTracker/artie.pesh-= imam/jobcache/job_201310301559_0024/jars/job.jar:/d04/dfs/dn/tasktracker/ta= skTracker/artie.pesh-imam/jobcache/job_201310301559_0024/attempt_2013103015= 59_0024_m_000000_0/work 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=3D/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.= p0.39/lib/hadoop-0.20-mapreduce/lib/native/Linux-amd64-64:/d04/dfs/dn/taskt= racker/taskTracker/artie.pesh-imam/jobcache/job_201310301559_0024/attempt_2= 01310301559_0024_m_000000_0/work 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=3D/d04/dfs/dn/tasktracker/taskTracker/artie.pesh= -imam/jobcache/job_201310301559_0024/attempt_201310301559_0024_m_000000_0/w= ork/tmp 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=3D 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=3DLinux 2013-10-= 30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=3Damd64 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=3D2.6.32-358.6.2.el6.x86_64 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=3Dmapred 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=3D/var/lib/hadoop-mapreduce 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=3D/d04/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/= jobcache/job_201310301559_0024/attempt_201310301559_0024_m_000000_0/work 2013-10-30 19:38:09,318 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=3Ddatanode05.prd.nj1.tapad.com:22181sessionTimeout=3D60000 watcher=3Dorg.apache.giraph.master.BspServiceMaster@20d145ec 2013-10-30 19:38:09,339 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server datanode05.prd.nj1.tapad.com/10.20.20.31:22181. Will not attempt to authenticate using SASL (unknown error) 2013-10-30 19:38:09,340 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to datanode05.prd.nj1.tapad.com/10.20.20.31:22181, initiating session 2013-10-30 19:38:09,413 WARN org.apache.zookeeper.ClientCnxnSocket: Connected to an old server; r-o mode will be unavailable 2013-10-30 19:38:09,413 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server datanode05.prd.nj1.tapad.com/10.20.20.31:22181, sessionid =3D 0x1420adf77390000, negotiated timeout =3D 600000 2013-10-30 19:38:09,415 INFO org.apache.giraph.bsp.BspService: process: Asynchronous connection complete. 2013-10-30 19:38:09,427 INFO org.apache.giraph.graph.GraphTaskManager: map: No need to do anything when not a worker 2013-10-30 19:38:09,427 INFO org.apache.giraph.graph.GraphTaskManager: cleanup: Starting for MASTER_ZOOKEEPER_ONLY 2013-10-30 19:38:09,522 INFO org.apache.giraph.master.BspServiceMaster: becomeMaster: First child is '/_hadoopBsp/job_201310301559_0024/_masterElectionDir/datanode05.prd.nj1.ta= pad.com_00000000000' and my bid is '/_hadoopBsp/job_201310301559_0024/_masterElectionDir/datanode05.prd.nj1.ta= pad.com_00000000000' 2013-10-30 19:38:09,605 INFO org.apache.giraph.comm.netty.NettyServer: NettyServer: Using execution handler with 8 threads after requestFrameDecoder. 2013-10-30 19:38:09,644 INFO org.apache.giraph.comm.netty.NettyServer: start: Started server communication server: datanode05.prd.nj1.tapad.com/10.20.20.31:30000 with up to 16 threads on bind attempt 0 with sendBufferSize =3D 32768 receiveBufferSize =3D 524288 backlog =3D 1 2013-10-30 19:38:09,653 INFO org.apache.giraph.comm.netty.NettyClient: NettyClient: Using execution handler with 8 threads after requestEncoder. 2013-10-30 19:38:09,659 INFO org.apache.giraph.master.BspServiceMaster: becomeMaster: I am now the master! 2013-10-30 19:38:09,708 INFO org.apache.giraph.bsp.BspService: process: applicationAttemptChanged signaled 2013-10-30 19:38:09,773 WARN org.apache.giraph.bsp.BspService: process: Unknown and unprocessed event (path=3D/_hadoopBsp/job_201310301559_0024/_applicationAttemptsDir/0/_supers= tepDir, type=3DNodeChildrenChanged, state=3DSyncConnected) 2013-10-30 19:38:39,793 = INFO org.apache.giraph.master.BspServiceMaster: checkWorkers: Only found 0 responses of -1 needed to start superstep -1. Reporting every 30000 msecs, 569875 more msecs left before giving up. 2013-10-30 19:39:09,824 INFO org.apache.giraph.master.BspServiceMaster: checkWorkers: Only found 0 responses of -1 needed to start superstep -1. Reporting every 30000 msecs, 539844 more msecs left before giving up. 2013-10-30 19:39:39,846 INFO org.apache.giraph.master.BspServiceMaster: checkWorkers: Only found 0 responses of -1 needed to start superstep -1. Reporting every 30000 msecs, 509822 more msecs left before giving up. 2013-10-30 19:40:09,869 INFO org.apache.giraph.master.BspServiceMaster: checkWorkers: Only found 0 responses of -1 needed to start superstep -1. Reporting every 30000 msecs, 479799 more msecs left before giving up. 2013-10-30 19:40:39,908 INFO org.apache.giraph.master.BspServiceMaster: checkWorkers: Only found 0 responses of -1 needed to start superstep -1. Reporting every 30000 msecs, 449760 more msecs left before giving up. 2013-10-30 19:41:09,931 INFO org.apache.giraph.master.BspServiceMaster: checkWorkers: Only found 0 responses of -1 needed to start superstep -1. Reporting every 30000 msecs, 419737 more msecs left before giving up. 2013-10-30 19:41:39,970 INFO org.apache.giraph.master.BspServiceMaster: checkWorkers: Only found 0 responses of -1 needed to start superstep -1. Reporting every 30000 msecs, 389698 more msecs left before giving up. 2013-10-30 19:42:10,001 INFO org.apache.giraph.master.BspServiceMaster: checkWorkers: Only found 0 responses of -1 needed to start superstep -1. Reporting every 30000 msecs, 359667 more msecs left before giving up. 2013-10-30 19:42:40,040 INFO org.apache.giraph.master.BspServiceMaster: checkWorkers: Only found 0 responses of -1 needed to start superstep -1. Reporting every 30000 msecs, 329628 more msecs left before giving up. 2013-10-30 19:43:10,062 INFO org.apache.giraph.master.BspServiceMaster: checkWorkers: Only found 0 responses of -1 needed to start superstep -1. Reporting every 30000 msecs, 299606 more msecs left before giving up. 2013-10-30 19:43:40,102 INFO org.apache.giraph.master.BspServiceMaster: checkWorkers: Only found 0 responses of -1 needed to start superstep -1. Reporting every 30000 msecs, 269566 more msecs left before giving up. --047d7bdc94ccd2aaa404e9fa9fc1 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
Hi all,

I'm able to run this job lo= cally but when I try to run against our cluster, the job seems to fail. Her= e's the log entries.

It looks like it's no= t proceeding beyond superstep 0. Is there anyway to get a more detailed pic= ture of what's going on?

2013-10-30 19:38:04,467 WARN mapreduce.Cou= nters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.a= pache.hadoop.mapreduce.TaskCounter instead 2013-10-30 19:38:05,157 WARN org.apache.hadoop.conf.Configuration: session.id is deprecated. Instead, use dfs.metri= cs.session-id 2013-10-30 19:38:05,158 INFO org.apache.hadoop.metrics.jvm.JvmMetrics: Init= ializing JVM Metrics with processName=3DMAP, sessionId=3D 2013-10-30 19:38:05,634 INFO org.apache.hadoop.util.ProcessTree: setsid exi= ted with exit code 0 2013-10-30 19:38:05,639 INFO org.apache.hadoop.mapred.Task: Using Resource= CalculatorPlugin : org.apache.hadoop.util.LinuxResourceCalculatorPlugin@371= e88fb 2013-10-30 19:38:06,029 INFO org.apache.hadoop.mapred.MapTask: Processing s= plit: 'org.apache.giraph.bsp.BspInputSplit, index=3D-1, num=3D-1 2013-10-30 19:38:06,047 INFO org.apache.giraph.graph.GraphTaskManager: setu= p: Log level remains at info 2013-10-30 19:38:06,117 INFO org.apache.giraph.graph.GraphTaskManager: Dist= ributed cache is empty. Assuming fatjar. 2013-10-30 19:38:06,117 INFO org.apache.giraph.graph.GraphTaskManager: setu= p: classpath @ /d02/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jobcache= /job_201310301559_0024/jars/job.jar for job jobs.ConnectedComponents 2013-10-30 19:38:06,155 INFO org.apache.giraph.zk.ZooKeeperManager: createC= andidateStamp: Made the directory _bsp/_defaultZkManagerDir/job_20131030155= 9_0024 2013-10-30 19:38:06,166 INFO org.apache.giraph.zk.ZooKeeperManager: createC= andidateStamp: Creating my filestamp _bsp/_defaultZkManagerDir/job_20131030= 1559_0024/_task/datanode05.= prd.nj1.tapad.com 0 2013-10-30 19:38:06,216 INFO org.apache.giraph.zk.ZooKeeperManager: getZooK= eeperServerList: Got [datan= ode05.prd.nj1.tapad.com] 1 hosts from 1 candidates when 1 required (pol= ling period is 3000) on attempt 0 2013-10-30 19:38:06,217 INFO org.apache.giraph.zk.ZooKeeperManager: createZ= ooKeeperServerList: Creating the final ZooKeeper file '_bsp/_defaultZkM= anagerDir/job_201310301559_0024/zkServerList_datanode05.prd.nj1.tapad.com 0 ' 2013-10-30 19:38:06,235 INFO org.apache.giraph.zk.ZooKeeperManager: getZooK= eeperServerList: For task 0, got file 'zkServerList_datanode05.prd.nj1.tapad.com = 0 ' (polling period is 3000) 2013-10-30 19:38:06,235 INFO org.apache.giraph.zk.ZooKeeperManager: getZooK= eeperServerList: Found [dat= anode05.prd.nj1.tapad.com, 0] 2 hosts in filename 'zkServerList_datanode05.prd.nj= 1.tapad.com 0 ' 2013-10-30 19:38:06,236 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZ= ooKeeperServers: Trying to delete old directory /d06/dfs/dn/tasktracker/tas= kTracker/artie.pesh-imam/jobcache/job_201310301559_0024/work/_bspZooKeeper 2013-10-30 19:38:06,243 INFO org.apache.giraph.zk.ZooKeeperManager: generat= eZooKeeperConfigFile: Creating file /d06/dfs/dn/tasktracker/taskTracker/art= ie.pesh-imam/jobcache/job_201310301559_0024/work/_bspZooKeeper/zoo.cfg in /= d06/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jobcache/job_20131030155= 9_0024/work/_bspZooKeeper with base port 22181 2013-10-30 19:38:06,243 INFO org.apache.giraph.zk.ZooKeeperManager: generat= eZooKeeperConfigFile: Make directory of _bspZooKeeper =3D true 2013-10-30 19:38:06,243 INFO org.apache.giraph.zk.ZooKeeperManager: generat= eZooKeeperConfigFile: Delete of zoo.cfg =3D false 2013-10-30 19:38:06,246 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZ= ooKeeperServers: Attempting to start ZooKeeper server with command [/usr/ja= va/jdk1.7.0_40/jre/bin/java, -Xmx512m, -XX:ParallelGCThreads=3D4, -XX:+UseC= oncMarkSweepGC, -XX:CMSInitiatingOccupancyFraction=3D70, -XX:MaxGCPauseMill= is=3D100, -cp, /d02/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jobcache= /job_201310301559_0024/jars/job.jar, org.apache.zookeeper.server.quorum.Quo= rumPeerMain, /d06/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jobcache/j= ob_201310301559_0024/work/_bspZooKeeper/zoo.cfg] in directory /d06/dfs/dn/t= asktracker/taskTracker/artie.pesh-imam/jobcache/job_201310301559_0024/work/= _bspZooKeeper 2013-10-30 19:38:06,249 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZ= ooKeeperServers: Shutdown hook added. 2013-10-30 19:38:06,249 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZ= ooKeeperServers: Connect attempt 0 of 10 max trying to connect to datanode05.prd.nj1.tapad.com= :22181 with poll msecs =3D 3000 2013-10-30 19:38:06,253 WARN org.apache.giraph.zk.ZooKeeperManager: onlineZ= ooKeeperServers: Got ConnectException java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java= :339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketIm= pl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:1= 82) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperM= anager.java:701) at org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTas= kManager.java:357) at org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:18= 8) at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:60) at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:90) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:672) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:330) at org.apache.hadoop.mapred.Child$4.run(Child.java:268) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformati= on.java:1408) at org.apache.hadoop.mapred.Child.main(Child.java:262) 2013-10-30 19:38:09,256 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZ= ooKeeperServers: Connect attempt 1 of 10 max trying to connect to datanode05.prd.nj1.tapad.com= :22181 with poll msecs =3D 3000 2013-10-30 19:38:09,256 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZ= ooKeeperServers: Connected to datanode05.prd.nj1.tapad.com/10.20.20.31:22181! 2013-10-30 19:38:09,256 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZ= ooKeeperServers: Creating my filestamp _bsp/_defaultZkManagerDir/job_201310= 301559_0024/_zkServer/datan= ode05.prd.nj1.tapad.com 0 2013-10-30 19:38:09,291 INFO org.apache.giraph.graph.GraphTaskManager: setu= p: Chosen to run ZooKeeper... 2013-10-30 19:38:09,291 INFO org.apache.giraph.graph.GraphTaskManager: setu= p: Starting up BspServiceMaster (master thread)... 2013-10-30 19:38:09,306 INFO org.apache.giraph.bsp.BspService: BspService: = Connecting to ZooKeeper with job job_201310301559_0024, 0 on datanode05.prd.nj1.tapad.com:22181<= /a> 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:zookeeper.version=3D3.4.5-cdh4.4.0--1, built on 09/03/2013 16:14 GMT 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:host.name=3Ddatanode05.prd.nj1.tapad.com 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:java.version=3D1.7.0_40 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:java.vendor=3DOracle Corporation 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:java.home=3D/usr/java/jdk1.7.0_40/jre 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:java.class.path=3D/var/run/cloudera-scm-agent/process/238-mapreduce-TA= SKTRACKER:/usr/java/jdk1.7.0_40/lib/tools.jar:/opt/cloudera/parcels/CDH-4.4= .0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce:/opt/cloudera/parcels/CDH-4.4= .0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/hadoop-core-2.0.0-mr1-cdh4.4.= 0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapr= educe/lib/activation-1.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.= 39/lib/hadoop-0.20-mapreduce/lib/ant-contrib-1.0b3.jar:/opt/cloudera/parcel= s/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/asm-3.2.jar:/opt= /cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/= avro-1.7.4.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-= 0.20-mapreduce/lib/avro-compiler-1.7.4.jar:/opt/cloudera/parcels/CDH-4.4.0-= 1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/commons-beanutils-1.7.0.jar:= /opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/= lib/commons-beanutils-core-1.8.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4= .4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/commons-cli-1.2.jar:/opt/cloudera/= parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/commons-co= dec-1.4.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.2= 0-mapreduce/lib/commons-collections-3.2.1.jar:/opt/cloudera/parcels/CDH-4.4= .0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/commons-compress-1.4.1.ja= r:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduc= e/lib/commons-configuration-1.6.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.= 4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/commons-digester-1.8.jar:/opt/cloud= era/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/common= s-el-1.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.= 20-mapreduce/lib/commons-httpclient-3.1.jar:/opt/cloudera/parcels/CDH-4.4.0= -1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/commons-io-2.1.jar:/opt/clo= udera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/comm= ons-lang-2.5.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoo= p-0.20-mapreduce/lib/commons-logging-1.1.1.jar:/opt/cloudera/parcels/CDH-4.= 4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/commons-math-2.1.jar:/op= t/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib= /commons-net-3.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/h= adoop-0.20-mapreduce/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1= .cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/hadoop-fairscheduler-2.0.0-mr= 1-cdh4.4.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-= 0.20-mapreduce/lib/hsqldb-1.8.0.10.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cd= h4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jackson-core-asl-1.8.8.jar:/opt/= cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/j= ackson-jaxrs-1.8.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib= /hadoop-0.20-mapreduce/lib/jackson-mapper-asl-1.8.8.jar:/opt/cloudera/parce= ls/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jackson-xc-1.8.= 8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapr= educe/lib/jasper-compiler-5.5.23.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4= .4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jasper-runtime-5.5.23.jar:/opt/clo= udera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jaxb= -api-2.2.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-= 0.20-mapreduce/lib/jaxb-impl-2.2.3-1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.= cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jersey-core-1.8.jar:/opt/cloud= era/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jersey= -json-1.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0= .20-mapreduce/lib/jersey-server-1.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.c= dh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jets3t-0.6.1.jar:/opt/cloudera/= parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jettison-1= .1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-map= reduce/lib/jetty-6.1.26.cloudera.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cd= h4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jetty-util-6.1.26.cloudera.2.jar= :/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce= /lib/jline-0.9.94.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/= hadoop-0.20-mapreduce/lib/jsch-0.1.42.jar:/opt/cloudera/parcels/CDH-4.4.0-1= .cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jsp-api-2.1.jar:/opt/cloudera= /parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jsr305-1.= 3.9.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-ma= preduce/lib/junit-4.8.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.3= 9/lib/hadoop-0.20-mapreduce/lib/kfs-0.2.2.jar:/opt/cloudera/parcels/CDH-4.4= .0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/kfs-0.3.jar:/opt/cloudera= /parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/log4j-1.2= .17.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-ma= preduce/lib/mockito-all-1.8.5.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.= 0.p0.39/lib/hadoop-0.20-mapreduce/lib/paranamer-2.3.jar:/opt/cloudera/parce= ls/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/protobuf-java-2= .4.0a.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-= mapreduce/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.= 0.p0.39/lib/hadoop-0.20-mapreduce/lib/slf4j-api-1.6.1.jar:/opt/cloudera/par= cels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/snappy-java-1= .0.4.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20= -mapreduce/lib/stax-api-1.0.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.= 0.p0.39/lib/hadoop-0.20-mapreduce/lib/xmlenc-0.52.jar:/opt/cloudera/parcels= /CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/xz-1.0.jar:/opt/c= loudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/zo= okeeper-3.4.5-cdh4.4.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39= /lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-2.1.jar:/opt/cloudera/parcels/CD= H-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-0.20-mapreduce/lib/jsp-2.1/jsp-api-2.1.= jar:/usr/share/cmf/lib/plugins/tt-instrumentation-4.7.2.jar:/usr/share/cmf/= lib/plugins/navigator-plugin-4.7.2-shaded.jar:/usr/share/cmf/lib/plugins/ev= ent-publish-4.7.2-shaded.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.= 39/lib/hadoop-hdfs/lib/commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-4.4.0-= 1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/jsp-api-2.1.jar:/opt/cloudera/parcels/= CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/jasper-runtime-5.5.23.jar:/o= pt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/jersey-c= ore-1.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdf= s/lib/guava-11.0.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib= /hadoop-hdfs/lib/log4j-1.2.17.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.= 0.p0.39/lib/hadoop-hdfs/lib/zookeeper-3.4.5-cdh4.4.0.jar:/opt/cloudera/parc= els/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/servlet-api-2.5.jar:/opt= /cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/commons-lo= gging-1.1.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop= -hdfs/lib/asm-3.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/= hadoop-hdfs/lib/jetty-util-6.1.26.cloudera.2.jar:/opt/cloudera/parcels/CDH-= 4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/jackson-mapper-asl-1.8.8.jar:/op= t/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/protobuf-= java-2.4.0a.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop= -hdfs/lib/commons-lang-2.5.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p= 0.39/lib/hadoop-hdfs/lib/jersey-server-1.8.jar:/opt/cloudera/parcels/CDH-4.= 4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/xmlenc-0.52.jar:/opt/cloudera/parc= els/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/commons-el-1.0.jar:/opt/= cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/commons-cod= ec-1.4.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs= /lib/commons-io-2.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/li= b/hadoop-hdfs/lib/jetty-6.1.26.cloudera.2.jar:/opt/cloudera/parcels/CDH-4.4= .0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/jline-0.9.94.jar:/opt/cloudera/parc= els/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/jackson-core-asl-1.8.8.j= ar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/lib/com= mons-daemon-1.0.3.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/= hadoop-hdfs/lib/jsr305-1.3.9.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0= .p0.39/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.4.0.jar:/opt/cloudera/parcels= /CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/hadoop-hdfs-2.0.0-cdh4.4.0-test= s.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop-hdfs/hado= op-hdfs.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib= /commons-cli-1.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/h= adoop/lib/jsp-api-2.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/= lib/hadoop/lib/jasper-runtime-5.5.23.jar:/opt/cloudera/parcels/CDH-4.4.0-1.= cdh4.4.0.p0.39/lib/hadoop/lib/jackson-xc-1.8.8.jar:/opt/cloudera/parcels/CD= H-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jettison-1.1.jar:/opt/cloudera/parc= els/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jersey-core-1.8.jar:/opt/clou= dera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/guava-11.0.2.jar:/op= t/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/log4j-1.2.17.j= ar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/commons-= collections-3.2.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/= hadoop/lib/zookeeper-3.4.5-cdh4.4.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.c= dh4.4.0.p0.39/lib/hadoop/lib/servlet-api-2.5.jar:/opt/cloudera/parcels/CDH-= 4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/commons-beanutils-1.7.0.jar:/opt/clou= dera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/commons-logging-1.1.= 1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/asm-3= .2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jett= y-util-6.1.26.cloudera.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.= 39/lib/hadoop/lib/jackson-mapper-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-4.= 4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jsch-0.1.42.jar:/opt/cloudera/parcels/C= DH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/protobuf-java-2.4.0a.jar:/opt/clou= dera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/commons-lang-2.5.jar= :/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/commons-ht= tpclient-3.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoo= p/lib/jersey-server-1.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.3= 9/lib/hadoop/lib/xmlenc-0.52.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0= .p0.39/lib/hadoop/lib/commons-digester-1.8.jar:/opt/cloudera/parcels/CDH-4.= 4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jaxb-api-2.2.2.jar:/opt/cloudera/parcel= s/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/slf4j-log4j12-1.6.1.jar:/opt/cl= oudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jersey-json-1.8.ja= r:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jackson-j= axrs-1.8.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/= lib/commons-el-1.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib= /hadoop/lib/commons-compress-1.4.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cd= h4.4.0.p0.39/lib/hadoop/lib/commons-codec-1.4.jar:/opt/cloudera/parcels/CDH= -4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/mockito-all-1.8.5.jar:/opt/cloudera/= parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/commons-io-2.1.jar:/opt/c= loudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/slf4j-api-1.6.1.j= ar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jetty-6.= 1.26.cloudera.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/ha= doop/lib/junit-4.8.2.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/l= ib/hadoop/lib/paranamer-2.3.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.= p0.39/lib/hadoop/lib/jline-0.9.94.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh= 4.4.0.p0.39/lib/hadoop/lib/avro-1.7.4.jar:/opt/cloudera/parcels/CDH-4.4.0-1= .cdh4.4.0.p0.39/lib/hadoop/lib/activation-1.1.jar:/opt/cloudera/parcels/CDH= -4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/snappy-java-1.0.4.1.jar:/opt/clouder= a/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/hue-plugins-2.5.0-cdh4.= 4.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/kfs= -0.3.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/co= mmons-math-2.1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/had= oop/lib/jackson-core-asl-1.8.8.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4= .0.p0.39/lib/hadoop/lib/commons-beanutils-core-1.8.0.jar:/opt/cloudera/parc= els/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/commons-net-3.1.jar:/opt/clou= dera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/stax-api-1.0.1.jar:/= opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jaxb-impl-2.= 2.3-1.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/x= z-1.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/c= ommons-configuration-1.6.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.= 39/lib/hadoop/lib/jsr305-1.3.9.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4= .0.p0.39/lib/hadoop/lib/jasper-compiler-5.5.23.jar:/opt/cloudera/parcels/CD= H-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/lib/jets3t-0.6.1.jar:/opt/cloudera/parc= els/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/hadoop-common-2.0.0-cdh4.4.0-test= s.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/hadoop-an= notations-2.0.0-cdh4.4.0.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.= 39/lib/hadoop/hadoop-auth.jar:/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0= .39/lib/hadoop/hadoop-auth-2.0.0-cdh4.4.0.jar:/opt/cloudera/parcels/CDH-4.4= .0-1.cdh4.4.0.p0.39/lib/hadoop/hadoop-common-2.0.0-cdh4.4.0.jar:/opt/cloude= ra/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/hadoop-common.jar:/opt/clo= udera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/lib/hadoop/hadoop-annotations.jar:= /d02/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jobcache/job_2013103015= 59_0024/jars/classes:/d02/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jo= bcache/job_201310301559_0024/jars/job.jar:/d04/dfs/dn/tasktracker/taskTrack= er/artie.pesh-imam/jobcache/job_201310301559_0024/attempt_201310301559_0024= _m_000000_0/work 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:java.library.path=3D/opt/cloudera/parcels/CDH-4.4.0-1.cdh4.4.0.p0.39/l= ib/hadoop-0.20-mapreduce/lib/native/Linux-amd64-64:/d04/dfs/dn/tasktracker/= taskTracker/artie.pesh-imam/jobcache/job_201310301559_0024/attempt_20131030= 1559_0024_m_000000_0/work 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:java.io.tmpdir=3D/d04/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/j= obcache/job_201310301559_0024/attempt_201310301559_0024_m_000000_0/work/tmp 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:java.compiler=3D<NA> 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:os.name=3DLinux 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:os.arch=3Damd64 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:os.version=3D2.6.32-358.6.2.el6.x86_64 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:user.name=3Dmapred 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:user.home=3D/var/lib/hadoop-mapreduce 2013-10-30 19:38:09,317 INFO org.apache.zookeeper.ZooKeeper: Client environ= ment:user.dir=3D/d04/dfs/dn/tasktracker/taskTracker/artie.pesh-imam/jobcach= e/job_201310301559_0024/attempt_201310301559_0024_m_000000_0/work 2013-10-30 19:38:09,318 INFO org.apache.zookeeper.ZooKeeper: Initiating cli= ent connection, connectString=3Ddatanode05.prd.nj1.tapad.com:22181 sessionTimeout=3D60000 wa= tcher=3Dorg.apache.giraph.master.BspServiceMaster@20d145ec 2013-10-30 19:38:09,339 INFO org.apache.zookeeper.ClientCnxn: Opening socke= t connection to server datanode05.prd.nj1.tapad.com/10.20.20.31:22181. Will not = attempt to authenticate using SASL (unknown error) 2013-10-30 19:38:09,340 INFO org.apache.zookeeper.ClientCnxn: Socket connec= tion established to datanode05.prd.nj1.tapad.com/10.20.20.31:22181, initiating s= ession 2013-10-30 19:38:09,413 WARN org.apache.zookeeper.ClientCnxnSocket: Connect= ed to an old server; r-o mode will be unavailable 2013-10-30 19:38:09,413 INFO org.apache.zookeeper.ClientCnxn: Session estab= lishment complete on server datanode05.prd.nj1.tapad.com/10.20.20.31:22181, sess= ionid =3D 0x1420adf77390000, negotiated timeout =3D 600000 2013-10-30 19:38:09,415 INFO org.apache.giraph.bsp.BspService: process: Asy= nchronous connection complete. 2013-10-30 19:38:09,427 INFO org.apache.giraph.graph.GraphTaskManager: map:= No need to do anything when not a worker 2013-10-30 19:38:09,427 INFO org.apache.giraph.graph.GraphTaskManager: clea= nup: Starting for MASTER_ZOOKEEPER_ONLY 2013-10-30 19:38:09,522 INFO org.apache.giraph.master.BspServiceMaster: bec= omeMaster: First child is '/_hadoopBsp/job_201310301559_0024/_masterEle= ctionDir/datanode05.prd.nj1.tapad.com_00000000000' and my bid is '/= _hadoopBsp/job_201310301559_0024/_masterElectionDir/datanode05.prd.nj1.tapa= d.com_00000000000' 2013-10-30 19:38:09,605 INFO org.apache.giraph.comm.netty.NettyServer: Nett= yServer: Using execution handler with 8 threads after requestFrameDecoder. 2013-10-30 19:38:09,644 INFO org.apache.giraph.comm.netty.NettyServer: star= t: Started server communication server: datanode05.prd.nj1.tapad.com/10.20.20.31:300= 00 with up to 16 threads on bind attempt 0 with sendBufferSize =3D 3276= 8 receiveBufferSize =3D 524288 backlog =3D 1 2013-10-30 19:38:09,653 INFO org.apache.giraph.comm.netty.NettyClient: Nett= yClient: Using execution handler with 8 threads after requestEncoder. 2013-10-30 19:38:09,659 INFO org.apache.giraph.master.BspServiceMaster: bec= omeMaster: I am now the master! 2013-10-30 19:38:09,708 INFO org.apache.giraph.bsp.BspService: process: app= licationAttemptChanged signaled 2013-10-30 19:38:09,773 WARN org.apache.giraph.bsp.BspService: process: Unk= nown and unprocessed event (path=3D/_hadoopBsp/job_201310301559_0024/_appli= cationAttemptsDir/0/_superstepDir, type=3DNodeChildrenChanged, state=3DSync= Connected) 2013-10-30 19:38:39,793 INFO org.apache.giraph.master.BspServiceMaster: che= ckWorkers: Only found 0 responses of -1 needed to start superstep -1. Repo= rting every 30000 msecs, 569875 more msecs left before giving up. 2013-10-30 19:39:09,824 INFO org.apache.giraph.master.BspServiceMaster: che= ckWorkers: Only found 0 responses of -1 needed to start superstep -1. Repo= rting every 30000 msecs, 539844 more msecs left before giving up. 2013-10-30 19:39:39,846 INFO org.apache.giraph.master.BspServiceMaster: che= ckWorkers: Only found 0 responses of -1 needed to start superstep -1. Repo= rting every 30000 msecs, 509822 more msecs left before giving up. 2013-10-30 19:40:09,869 INFO org.apache.giraph.master.BspServiceMaster: che= ckWorkers: Only found 0 responses of -1 needed to start superstep -1. Repo= rting every 30000 msecs, 479799 more msecs left before giving up. 2013-10-30 19:40:39,908 INFO org.apache.giraph.master.BspServiceMaster: che= ckWorkers: Only found 0 responses of -1 needed to start superstep -1. Repo= rting every 30000 msecs, 449760 more msecs left before giving up. 2013-10-30 19:41:09,931 INFO org.apache.giraph.master.BspServiceMaster: che= ckWorkers: Only found 0 responses of -1 needed to start superstep -1. Repo= rting every 30000 msecs, 419737 more msecs left before giving up. 2013-10-30 19:41:39,970 INFO org.apache.giraph.master.BspServiceMaster: che= ckWorkers: Only found 0 responses of -1 needed to start superstep -1. Repo= rting every 30000 msecs, 389698 more msecs left before giving up. 2013-10-30 19:42:10,001 INFO org.apache.giraph.master.BspServiceMaster: che= ckWorkers: Only found 0 responses of -1 needed to start superstep -1. Repo= rting every 30000 msecs, 359667 more msecs left before giving up. 2013-10-30 19:42:40,040 INFO org.apache.giraph.master.BspServiceMaster: che= ckWorkers: Only found 0 responses of -1 needed to start superstep -1. Repo= rting every 30000 msecs, 329628 more msecs left before giving up. 2013-10-30 19:43:10,062 INFO org.apache.giraph.master.BspServiceMaster: che= ckWorkers: Only found 0 responses of -1 needed to start superstep -1. Repo= rting every 30000 msecs, 299606 more msecs left before giving up. 2013-10-30 19:43:40,102 INFO org.apache.giraph.master.BspServiceMaster: che= ckWorkers: Only found 0 responses of -1 needed to start superstep -1. Repo= rting every 30000 msecs, 269566 more msecs left before giving up.



--047d7bdc94ccd2aaa404e9fa9fc1--