giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavian Jacquot <tki1...@gmail.com>
Subject Re: java.lang.IllegalStateException SimpleShortestPaths
Date Wed, 28 Jan 2015 14:24:13 GMT
Hi,

I've noticied that you used only one worker in your command ( -w 1).
Have you tried to set the option giraph.SplitMasterWorker to false ?
I am pretty new to Giraph but from what I know, you have to do that when
you have only 1 worker.

Yours Sincerely, Flavian

2015-01-28 14:57 GMT+01:00 Fredrik Einarsson <einarsson.f@gmail.com>:

> Dear Giraph users,
>
> I am trying to get my Giraph installation to work but I am stuck.
>
> Below I paste my command and job tracker log entry. I appreciate your
> help.
>
> hadoop jar
> /usr/local/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar
> org.apache.giraph.GiraphRunner
> org.apache.giraph.examples.SimpleShortestPathsComputation -vif
> org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
> -vip /user/fredrik/input/tiny_graph.txt -vof
> org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
> /user/fredrik/output/shortestpaths -w 1
>
> Task Logs: 'attempt_201501281117_0016_m_000000_0'
>
> *stdout logs*
> ------------------------------
>
>
> *stderr logs*
>
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [file:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in [jar:file:/usr/local/hadoop/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
> SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
>
> ------------------------------
>
>
> *syslog logs*
>
> 2015-01-28 14:49:28,422 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop
library
> 2015-01-28 14:49:30,078 WARN org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source
name ugi already exists!
> 2015-01-28 14:49:30,745 INFO org.apache.hadoop.util.ProcessTree: setsid exited with exit
code 0
> 2015-01-28 14:49:30,781 INFO org.apache.hadoop.mapred.Task:  Using ResourceCalculatorPlugin
: org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2bf8e35d
> 2015-01-28 14:49:31,022 INFO org.apache.hadoop.mapred.MapTask: Processing split: 'org.apache.giraph.bsp.BspInputSplit,
index=-1, num=-1
> 2015-01-28 14:49:31,270 INFO org.apache.giraph.graph.GraphTaskManager: setup: Log level
remains at info
> 2015-01-28 14:49:31,852 INFO org.apache.giraph.zk.ZooKeeperManager: createCandidateStamp:
Made the directory _bsp/_defaultZkManagerDir/job_201501281117_0016
> 2015-01-28 14:49:31,854 INFO org.apache.giraph.zk.ZooKeeperManager: createCandidateStamp:
Made the directory _bsp/_defaultZkManagerDir/job_201501281117_0016/_zkServer
> 2015-01-28 14:49:31,857 INFO org.apache.giraph.zk.ZooKeeperManager: createCandidateStamp:
Creating my filestamp _bsp/_defaultZkManagerDir/job_201501281117_0016/_task/fredrik-virtualbox
0
> 2015-01-28 14:49:31,925 INFO org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList:
Got [fredrik-virtualbox] 1 hosts from 1 candidates when 1 required (polling period is 3000)
on attempt 0
> 2015-01-28 14:49:31,926 INFO org.apache.giraph.zk.ZooKeeperManager: createZooKeeperServerList:
Creating the final ZooKeeper file '_bsp/_defaultZkManagerDir/job_201501281117_0016/zkServerList_fredrik-virtualbox
0 '
> 2015-01-28 14:49:31,955 INFO org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList:
For task 0, got file 'zkServerList_fredrik-virtualbox 0 ' (polling period is 3000)
> 2015-01-28 14:49:31,955 INFO org.apache.giraph.zk.ZooKeeperManager: getZooKeeperServerList:
Found [fredrik-virtualbox, 0] 2 hosts in filename 'zkServerList_fredrik-virtualbox 0 '
> 2015-01-28 14:49:31,956 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers:
Trying to delete old directory /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper
> 2015-01-28 14:49:31,969 INFO org.apache.giraph.zk.ZooKeeperManager: generateZooKeeperConfigFile:
Creating file /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/zoo.cfg
in /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper
with base port 22181
> 2015-01-28 14:49:31,970 INFO org.apache.giraph.zk.ZooKeeperManager: generateZooKeeperConfigFile:
Make directory of _bspZooKeeper = true
> 2015-01-28 14:49:31,970 INFO org.apache.giraph.zk.ZooKeeperManager: generateZooKeeperConfigFile:
Delete of zoo.cfg = false
> 2015-01-28 14:49:31,989 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers:
Connect attempt 0 of 10 max trying to connect to fredrik-virtualbox:22181 with poll msecs
= 3000
> 2015-01-28 14:49:32,000 WARN org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers:
Got ConnectException
> java.net.ConnectException: Connection refused
> 	at java.net.PlainSocketImpl.socketConnect(Native Method)
> 	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
> 	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
> 	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
> 	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
> 	at java.net.Socket.connect(Socket.java:579)
> 	at org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:601)
> 	at org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:433)
> 	at org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:231)
> 	at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:55)
> 	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
> 	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
> 	at org.apache.hadoop.mapred.Child.main(Child.java:249)
> 2015-01-28 14:49:32,018 INFO org.apache.zookeeper.server.quorum.QuorumPeerConfig: Reading
configuration from: /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/zoo.cfg
> 2015-01-28 14:49:32,029 INFO org.apache.zookeeper.server.DatadirCleanupManager: autopurge.snapRetainCount
set to 3
> 2015-01-28 14:49:32,029 INFO org.apache.zookeeper.server.DatadirCleanupManager: autopurge.purgeInterval
set to 0
> 2015-01-28 14:49:32,029 INFO org.apache.zookeeper.server.DatadirCleanupManager: Purge
task is not scheduled.
> 2015-01-28 14:49:32,035 WARN org.apache.giraph.zk.InProcessZooKeeperRunner: Either no
config or no quorum defined in config, running  in standalone mode
> 2015-01-28 14:49:32,154 INFO org.apache.zookeeper.server.quorum.QuorumPeerConfig: Reading
configuration from: /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/zoo.cfg
> 2015-01-28 14:49:32,154 INFO org.apache.zookeeper.server.ZooKeeperServerMain: Starting
server
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:zookeeper.version=3.4.5-1392090,
built on 09/30/2012 17:52 GMT
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:host.name=fredrik-VirtualBox
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:java.version=1.7.0_65
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:java.vendor=Oracle
Corporation
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:java.class.path=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars/classes:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work:/usr/local/hadoop/libexec/../conf:/usr/lib/jvm/java-7-openjdk-amd64/lib/tools.jar:/usr/local/hadoop/libexec/..:/usr/local/hadoop/libexec/../hadoop-core-1.2.1.jar:/usr/local/hadoop/libexec/../lib/asm-3.2.jar:/usr/local/hadoop/libexec/../lib/aspectjrt-1.6.11.jar:/usr/local/hadoop/libexec/../lib/aspectjtools-1.6.11.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop/libexec/../lib/commons-cli-1.2.jar:/usr/local/hadoop/libexec/../lib/commons-codec-1.4.jar:/usr/local/hadoop/libexec/../lib/commons-collections-3.2.1.jar:/usr/local/hadoop/libexec/../lib/commons-configuration-1.6.jar:/usr/local/hadoop/libexec/../lib/commons-daemon-1.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-digester-1.8.jar:/usr/local/hadoop/libexec/../lib/commons-el-1.0.jar:/usr/local/hadoop/libexec/../lib/commons-httpclient-3.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-io-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-lang-2.4.jar:/usr/local/hadoop/libexec/../lib/commons-logging-1.1.1.jar:/usr/local/hadoop/libexec/../lib/commons-logging-api-1.0.4.jar:/usr/local/hadoop/libexec/../lib/commons-math-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-net-3.1.jar:/usr/local/hadoop/libexec/../lib/core-3.1.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-capacity-scheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-fairscheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-thriftfs-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop/libexec/../lib/jackson-core-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jasper-compiler-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jasper-runtime-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jdeb-0.8.jar:/usr/local/hadoop/libexec/../lib/jersey-core-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-json-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-server-1.8.jar:/usr/local/hadoop/libexec/../lib/jets3t-0.6.1.jar:/usr/local/hadoop/libexec/../lib/jetty-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jetty-util-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jsch-0.1.42.jar:/usr/local/hadoop/libexec/../lib/junit-4.5.jar:/usr/local/hadoop/libexec/../lib/kfs-0.2.2.jar:/usr/local/hadoop/libexec/../lib/log4j-1.2.15.jar:/usr/local/hadoop/libexec/../lib/mockito-all-1.8.5.jar:/usr/local/hadoop/libexec/../lib/oro-2.0.8.jar:/usr/local/hadoop/libexec/../lib/servlet-api-2.5-20081211.jar:/usr/local/hadoop/libexec/../lib/slf4j-api-1.4.3.jar:/usr/local/hadoop/libexec/../lib/slf4j-log4j12-1.4.3.jar:/usr/local/hadoop/libexec/../lib/xmlenc-0.52.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-api-2.1.jar
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:java.library.path=/usr/local/hadoop/libexec/../lib/native/Linux-amd64-64:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:java.io.tmpdir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work/tmp
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:java.compiler=<NA>
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:os.name=Linux
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:os.arch=amd64
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:os.version=3.16.0-29-generic
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:user.name=fredrik
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:user.home=/home/fredrik
> 2015-01-28 14:49:32,183 INFO org.apache.zookeeper.server.ZooKeeperServer: Server environment:user.dir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work
> 2015-01-28 14:49:32,215 INFO org.apache.zookeeper.server.ZooKeeperServer: tickTime set
to 6000
> 2015-01-28 14:49:32,215 INFO org.apache.zookeeper.server.ZooKeeperServer: minSessionTimeout
set to 600000
> 2015-01-28 14:49:32,216 INFO org.apache.zookeeper.server.ZooKeeperServer: maxSessionTimeout
set to 900000
> 2015-01-28 14:49:32,266 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: binding
to port 0.0.0.0/0.0.0.0:22181
> 2015-01-28 <http://0.0.0.0/0.0.0.0:221812015-01-28> 14:49:32,337 INFO org.apache.zookeeper.server.persistence.FileTxnSnapLog:
Snapshotting: 0x0 to /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/version-2/snapshot.0
> 2015-01-28 14:49:32,414 INFO org.apache.zookeeper.server.PrepRequestProcessor: zookeeper.skipACL=="yes",
ACL checks will be skipped
> 2015-01-28 14:49:35,030 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers:
Connect attempt 1 of 10 max trying to connect to fredrik-virtualbox:22181 with poll msecs
= 3000
> 2015-01-28 14:49:35,030 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers:
Connected to fredrik-VirtualBox/192.168.56.10:22181!
> 2015-01-28 14:49:35,031 INFO org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers:
Creating my filestamp _bsp/_defaultZkManagerDir/job_201501281117_0016/_zkServer/fredrik-virtualbox
0
> 2015-01-28 14:49:35,031 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted
socket connection from /192.168.56.10:34203
> 2015-01-28 14:49:35,798 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted
socket connection from /192.168.56.10:34204
> 2015-01-28 14:49:35,804 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting
to establish new session at /192.168.56.10:34204
> 2015-01-28 14:49:35,807 INFO org.apache.zookeeper.server.persistence.FileTxnLog: Creating
new log file: log.1
> 2015-01-28 14:49:35,825 INFO org.apache.zookeeper.server.ZooKeeperServer: Established
session 0x14b30cddb450000 with negotiated timeout 600000 for client /192.168.56.10:34204
> 2015-01-28 14:49:35,989 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x1 zxid:0x2
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode
= NoNode for /_hadoopBsp/job_201501281117_0016
> 2015-01-28 14:49:36,012 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x8 zxid:0x7
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0
Error:KeeperErrorCode = NoNode for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0
> 2015-01-28 14:49:36,013 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x9 zxid:0x8
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp Error:KeeperErrorCode = NodeExists for /_hadoopBsp
> 2015-01-28 14:49:36,013 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0xa zxid:0x9
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode
= NodeExists for /_hadoopBsp/job_201501281117_0016
> 2015-01-28 14:49:36,014 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0xb zxid:0xa
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir
Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir
> 2015-01-28 14:49:36,023 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0xf zxid:0xd
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
Error:KeeperErrorCode = NoNode for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
> 2015-01-28 14:49:36,025 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x10 zxid:0xe
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp Error:KeeperErrorCode = NodeExists for /_hadoopBsp
> 2015-01-28 14:49:36,027 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x11 zxid:0xf
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode
= NodeExists for /_hadoopBsp/job_201501281117_0016
> 2015-01-28 14:49:36,028 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x12 zxid:0x10
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir
Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir
> 2015-01-28 14:49:36,029 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x13 zxid:0x11
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0
Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0
> 2015-01-28 14:49:36,030 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x14 zxid:0x12
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir
Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir
> 2015-01-28 14:49:36,415 WARN org.apache.zookeeper.server.NIOServerCnxn: caught end of
stream exception
> EndOfStreamException: Unable to read additional data from client sessionid 0x0, likely
client has closed socket
> 	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
> 	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
> 	at java.lang.Thread.run(Thread.java:745)
> 2015-01-28 14:49:36,415 INFO org.apache.zookeeper.server.NIOServerCnxn: Closed socket
connection for client /192.168.56.10:34203 (no session established for client)
> 2015-01-28 14:49:37,361 INFO org.apache.giraph.graph.GraphTaskManager: setup: Chosen
to run ZooKeeper...
> 2015-01-28 14:49:37,362 INFO org.apache.giraph.graph.GraphTaskManager: setup: Starting
up BspServiceMaster (master thread)...
> 2015-01-28 14:49:37,374 INFO org.apache.giraph.bsp.BspService: BspService: Path to create
to halt is /_hadoopBsp/job_201501281117_0016/_haltComputation
> 2015-01-28 14:49:37,374 INFO org.apache.giraph.bsp.BspService: BspService: Connecting
to ZooKeeper with job job_201501281117_0016, 0 on fredrik-virtualbox:22181
> 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client environment:zookeeper.version=3.4.5-1392090,
built on 09/30/2012 17:52 GMT
> 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client environment:host.name=fredrik-VirtualBox
> 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.version=1.7.0_65
> 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.vendor=Oracle
Corporation
> 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
> 2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.class.path=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars/classes:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work:/usr/local/hadoop/libexec/../conf:/usr/lib/jvm/java-7-openjdk-amd64/lib/tools.jar:/usr/local/hadoop/libexec/..:/usr/local/hadoop/libexec/../hadoop-core-1.2.1.jar:/usr/local/hadoop/libexec/../lib/asm-3.2.jar:/usr/local/hadoop/libexec/../lib/aspectjrt-1.6.11.jar:/usr/local/hadoop/libexec/../lib/aspectjtools-1.6.11.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop/libexec/../lib/commons-cli-1.2.jar:/usr/local/hadoop/libexec/../lib/commons-codec-1.4.jar:/usr/local/hadoop/libexec/../lib/commons-collections-3.2.1.jar:/usr/local/hadoop/libexec/../lib/commons-configuration-1.6.jar:/usr/local/hadoop/libexec/../lib/commons-daemon-1.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-digester-1.8.jar:/usr/local/hadoop/libexec/../lib/commons-el-1.0.jar:/usr/local/hadoop/libexec/../lib/commons-httpclient-3.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-io-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-lang-2.4.jar:/usr/local/hadoop/libexec/../lib/commons-logging-1.1.1.jar:/usr/local/hadoop/libexec/../lib/commons-logging-api-1.0.4.jar:/usr/local/hadoop/libexec/../lib/commons-math-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-net-3.1.jar:/usr/local/hadoop/libexec/../lib/core-3.1.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-capacity-scheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-fairscheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-thriftfs-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop/libexec/../lib/jackson-core-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jasper-compiler-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jasper-runtime-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jdeb-0.8.jar:/usr/local/hadoop/libexec/../lib/jersey-core-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-json-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-server-1.8.jar:/usr/local/hadoop/libexec/../lib/jets3t-0.6.1.jar:/usr/local/hadoop/libexec/../lib/jetty-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jetty-util-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jsch-0.1.42.jar:/usr/local/hadoop/libexec/../lib/junit-4.5.jar:/usr/local/hadoop/libexec/../lib/kfs-0.2.2.jar:/usr/local/hadoop/libexec/../lib/log4j-1.2.15.jar:/usr/local/hadoop/libexec/../lib/mockito-all-1.8.5.jar:/usr/local/hadoop/libexec/../lib/oro-2.0.8.jar:/usr/local/hadoop/libexec/../lib/servlet-api-2.5-20081211.jar:/usr/local/hadoop/libexec/../lib/slf4j-api-1.4.3.jar:/usr/local/hadoop/libexec/../lib/slf4j-log4j12-1.4.3.jar:/usr/local/hadoop/libexec/../lib/xmlenc-0.52.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-api-2.1.jar
> 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.library.path=/usr/local/hadoop/libexec/../lib/native/Linux-amd64-64:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work
> 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.io.tmpdir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work/tmp
> 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:java.compiler=<NA>
> 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.name=Linux
> 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.arch=amd64
> 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:os.version=3.16.0-29-generic
> 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.name=fredrik
> 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.home=/home/fredrik
> 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client environment:user.dir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work
> 2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection,
connectString=fredrik-virtualbox:22181 sessionTimeout=60000 watcher=org.apache.giraph.master.BspServiceMaster@f9aac17
> 2015-01-28 14:49:37,393 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection
to server fredrik-VirtualBox/192.168.56.10:22181. Will not attempt to authenticate using SASL
(unknown error)
> 2015-01-28 14:49:37,393 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted
socket connection from /192.168.56.10:34205
> 2015-01-28 14:49:37,394 INFO org.apache.zookeeper.ClientCnxn: Socket connection established
to fredrik-VirtualBox/192.168.56.10:22181, initiating session
> 2015-01-28 14:49:37,394 INFO org.apache.zookeeper.server.ZooKeeperServer: Client attempting
to establish new session at /192.168.56.10:34205
> 2015-01-28 14:49:37,395 INFO org.apache.zookeeper.server.ZooKeeperServer: Established
session 0x14b30cddb450001 with negotiated timeout 600000 for client /192.168.56.10:34205
> 2015-01-28 14:49:37,395 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete
on server fredrik-VirtualBox/192.168.56.10:22181, sessionid = 0x14b30cddb450001, negotiated
timeout = 600000
> 2015-01-28 14:49:37,396 INFO org.apache.giraph.bsp.BspService: process: Asynchronous
connection complete.
> 2015-01-28 14:49:37,402 INFO org.apache.giraph.graph.GraphTaskManager: map: No need to
do anything when not a worker
> 2015-01-28 14:49:37,402 INFO org.apache.giraph.graph.GraphTaskManager: cleanup: Starting
for MASTER_ZOOKEEPER_ONLY
> 2015-01-28 14:49:37,404 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x1 zxid:0x17
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_masterElectionDir Error:KeeperErrorCode
= NoNode for /_hadoopBsp/job_201501281117_0016/_masterElectionDir
> 2015-01-28 14:49:37,405 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x2 zxid:0x18
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp Error:KeeperErrorCode = NodeExists for /_hadoopBsp
> 2015-01-28 14:49:37,406 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x3 zxid:0x19
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode
= NodeExists for /_hadoopBsp/job_201501281117_0016
> 2015-01-28 14:49:37,409 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x6 zxid:0x1c
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_masterJobState Error:KeeperErrorCode
= NodeExists for /_hadoopBsp/job_201501281117_0016/_masterJobState
> 2015-01-28 14:49:37,409 INFO org.apache.giraph.bsp.BspService: getJobState: Job state
already exists (/_hadoopBsp/job_201501281117_0016/_masterJobState)
> 2015-01-28 14:49:37,411 INFO org.apache.giraph.master.BspServiceMaster: becomeMaster:
First child is '/_hadoopBsp/job_201501281117_0016/_masterElectionDir/fredrik-virtualbox_00000000000'
and my bid is '/_hadoopBsp/job_201501281117_0016/_masterElectionDir/fredrik-virtualbox_00000000000'
> 2015-01-28 14:49:37,414 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x9 zxid:0x1d
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir
Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir
> 2015-01-28 14:49:37,414 INFO org.apache.giraph.bsp.BspService: getApplicationAttempt:
Node /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir already exists!
> 2015-01-28 14:49:37,477 INFO org.apache.giraph.comm.netty.NettyServer: NettyServer: Using
execution group with 8 threads for requestFrameDecoder.
> 2015-01-28 14:49:37,537 INFO org.apache.giraph.comm.netty.NettyServer: start: Started
server communication server: fredrik-VirtualBox/192.168.56.10:30000 with up to 16 threads
on bind attempt 0 with sendBufferSize = 32768 receiveBufferSize = 524288
> 2015-01-28 14:49:37,545 INFO org.apache.giraph.comm.netty.NettyClient: NettyClient: Using
execution handler with 8 threads after request-encoder.
> 2015-01-28 14:49:37,548 INFO org.apache.giraph.master.BspServiceMaster: becomeMaster:
I am now the master!
> 2015-01-28 14:49:37,549 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0xb zxid:0x1e
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir
Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir
> 2015-01-28 14:49:37,549 INFO org.apache.giraph.bsp.BspService: getApplicationAttempt:
Node /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir already exists!
> 2015-01-28 14:49:37,550 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0xd zxid:0x1f
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
> 2015-01-28 14:49:37,559 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x13 zxid:0x21
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
> 2015-01-28 14:49:37,560 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x14 zxid:0x22
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir
Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir
> 2015-01-28 14:49:37,568 INFO org.apache.giraph.io.formats.GiraphFileInputFormat: Total
input paths to process : 1
> 2015-01-28 14:49:37,580 WARN org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy
native library not loaded
> 2015-01-28 14:49:37,586 INFO org.apache.giraph.master.BspServiceMaster: generateVertexInputSplits:
Got 1 input splits for 1 input threads
> 2015-01-28 14:49:37,586 INFO org.apache.giraph.master.BspServiceMaster: createVertexInputSplits:
Starting to write input split data to zookeeper with 1 threads
> 2015-01-28 14:49:37,590 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x18 zxid:0x23
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_vertexInputSplitDir Error:KeeperErrorCode
= NoNode for /_hadoopBsp/job_201501281117_0016/_vertexInputSplitDir
> 2015-01-28 14:49:37,591 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x19 zxid:0x24
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp Error:KeeperErrorCode = NodeExists for /_hadoopBsp
> 2015-01-28 14:49:37,591 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x1a zxid:0x25
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode
= NodeExists for /_hadoopBsp/job_201501281117_0016
> 2015-01-28 14:49:37,594 INFO org.apache.giraph.master.BspServiceMaster: createVertexInputSplits:
Done writing input split data to zookeeper
> 2015-01-28 14:49:37,595 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x1e zxid:0x29
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
> 2015-01-28 14:49:37,596 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001 type:create cxid:0x1f zxid:0x2a
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir
Error:KeeperErrorCode = NodeExists for /_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir
> 2015-01-28 14:49:37,605 INFO org.apache.giraph.comm.netty.NettyClient: Using Netty without
authentication.
> 2015-01-28 14:49:37,614 INFO org.apache.giraph.comm.netty.NettyClient: connectAllAddresses:
Successfully added 1 connections, (1 total connected) 0 failed, 0 failures total.
> 2015-01-28 14:49:37,615 INFO org.apache.giraph.partition.PartitionUtils: computePartitionCount:
Creating 1, default would have been 1 partitions.
> 2015-01-28 14:49:37,679 INFO org.apache.giraph.master.BspServiceMaster: barrierOnWorkerList:
0 out of 1 workers finished on superstep -1 on path /_hadoopBsp/job_201501281117_0016/_vertexInputSplitDoneDir
> 2015-01-28 14:49:37,681 INFO org.apache.giraph.comm.netty.NettyServer: start: Using Netty
without authentication.
> 2015-01-28 14:49:37,683 INFO org.apache.giraph.master.BspServiceMaster: barrierOnWorkerList:
Waiting on [fredrik-virtualbox_1]
> 2015-01-28 14:49:37,779 INFO org.apache.giraph.master.BspServiceMaster: setJobState:
{"_stateKey":"FAILED","_applicationAttemptKey":-1,"_superstepKey":-1} on superstep -1
> 2015-01-28 14:49:37,782 INFO org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000 type:create cxid:0x24 zxid:0x32
txntype:-1 reqpath:n/a Error Path:/_hadoopBsp/job_201501281117_0016/_masterJobState Error:KeeperErrorCode
= NodeExists for /_hadoopBsp/job_201501281117_0016/_masterJobState
> 2015-01-28 14:49:37,785 INFO org.apache.giraph.master.BspServiceMaster: setJobState:
{"_stateKey":"FAILED","_applicationAttemptKey":-1,"_superstepKey":-1}
> 2015-01-28 14:49:37,785 FATAL org.apache.giraph.master.BspServiceMaster: failJob: Killing
job job_201501281117_0016
> 2015-01-28 14:49:37,785 FATAL org.apache.giraph.master.BspServiceMaster: failJob: exception
java.lang.IllegalStateException: ******* WORKERS [Worker(hostname=fredrik-virtualbox, MRtaskID=1,
port=30001)] FAILED *******
> 2015-01-28 14:49:37,830 ERROR org.apache.giraph.master.MasterThread: masterThread: Master
algorithm failed with IllegalStateException
> java.lang.IllegalStateException: coordinateVertexInputSplits: Worker failed during input
split (currently not supported)
> 	at org.apache.giraph.master.BspServiceMaster.coordinateInputSplits(BspServiceMaster.java:1499)
> 	at org.apache.giraph.master.BspServiceMaster.coordinateSuperstep(BspServiceMaster.java:1666)
> 	at org.apache.giraph.master.MasterThread.run(MasterThread.java:124)
> 2015-01-28 14:49:37,831 FATAL org.apache.giraph.graph.GraphTaskManager: uncaughtException:
OverrideExceptionHandler on thread org.apache.giraph.master.MasterThread, msg = java.lang.IllegalStateException:
coordinateVertexInputSplits: Worker failed during input split (currently not supported), exiting...
> java.lang.IllegalStateException: java.lang.IllegalStateException: coordinateVertexInputSplits:
Worker failed during input split (currently not supported)
> 	at org.apache.giraph.master.MasterThread.run(MasterThread.java:194)
> Caused by: java.lang.IllegalStateException: coordinateVertexInputSplits: Worker failed
during input split (currently not supported)
> 	at org.apache.giraph.master.BspServiceMaster.coordinateInputSplits(BspServiceMaster.java:1499)
> 	at org.apache.giraph.master.BspServiceMaster.coordinateSuperstep(BspServiceMaster.java:1666)
> 	at org.apache.giraph.master.MasterThread.run(MasterThread.java:124)
>
>
>
> *Kinds Regards*
> Fredrik
>

Mime
View raw message