giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fredrik Einarsson <einarsso...@gmail.com>
Subject java.lang.IllegalStateException SimpleShortestPaths
Date Wed, 28 Jan 2015 13:57:32 GMT
Dear Giraph users,

I am trying to get my Giraph installation to work but I am stuck.

Below I paste my command and job tracker log entry. I appreciate your help.

hadoop jar
/usr/local/giraph/giraph-examples/target/giraph-examples-1.2.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar
org.apache.giraph.GiraphRunner
org.apache.giraph.examples.SimpleShortestPathsComputation -vif
org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat
-vip /user/fredrik/input/tiny_graph.txt -vof
org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op
/user/fredrik/output/shortestpaths -w 1

Task Logs: 'attempt_201501281117_0016_m_000000_0'

*stdout logs*
------------------------------


*stderr logs*

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in
[file:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/local/hadoop/lib/slf4j-log4j12-1.4.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]

------------------------------


*syslog logs*

2015-01-28 14:49:28,422 INFO org.apache.hadoop.util.NativeCodeLoader:
Loaded the native-hadoop library
2015-01-28 14:49:30,078 WARN
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi
already exists!
2015-01-28 14:49:30,745 INFO org.apache.hadoop.util.ProcessTree:
setsid exited with exit code 0
2015-01-28 14:49:30,781 INFO org.apache.hadoop.mapred.Task:  Using
ResourceCalculatorPlugin :
org.apache.hadoop.util.LinuxResourceCalculatorPlugin@2bf8e35d
2015-01-28 14:49:31,022 INFO org.apache.hadoop.mapred.MapTask:
Processing split: 'org.apache.giraph.bsp.BspInputSplit, index=-1,
num=-1
2015-01-28 14:49:31,270 INFO org.apache.giraph.graph.GraphTaskManager:
setup: Log level remains at info
2015-01-28 14:49:31,852 INFO org.apache.giraph.zk.ZooKeeperManager:
createCandidateStamp: Made the directory
_bsp/_defaultZkManagerDir/job_201501281117_0016
2015-01-28 14:49:31,854 INFO org.apache.giraph.zk.ZooKeeperManager:
createCandidateStamp: Made the directory
_bsp/_defaultZkManagerDir/job_201501281117_0016/_zkServer
2015-01-28 14:49:31,857 INFO org.apache.giraph.zk.ZooKeeperManager:
createCandidateStamp: Creating my filestamp
_bsp/_defaultZkManagerDir/job_201501281117_0016/_task/fredrik-virtualbox
0
2015-01-28 14:49:31,925 INFO org.apache.giraph.zk.ZooKeeperManager:
getZooKeeperServerList: Got [fredrik-virtualbox] 1 hosts from 1
candidates when 1 required (polling period is 3000) on attempt 0
2015-01-28 14:49:31,926 INFO org.apache.giraph.zk.ZooKeeperManager:
createZooKeeperServerList: Creating the final ZooKeeper file
'_bsp/_defaultZkManagerDir/job_201501281117_0016/zkServerList_fredrik-virtualbox
0 '
2015-01-28 14:49:31,955 INFO org.apache.giraph.zk.ZooKeeperManager:
getZooKeeperServerList: For task 0, got file
'zkServerList_fredrik-virtualbox 0 ' (polling period is 3000)
2015-01-28 14:49:31,955 INFO org.apache.giraph.zk.ZooKeeperManager:
getZooKeeperServerList: Found [fredrik-virtualbox, 0] 2 hosts in
filename 'zkServerList_fredrik-virtualbox 0 '
2015-01-28 14:49:31,956 INFO org.apache.giraph.zk.ZooKeeperManager:
onlineZooKeeperServers: Trying to delete old directory
/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper
2015-01-28 14:49:31,969 INFO org.apache.giraph.zk.ZooKeeperManager:
generateZooKeeperConfigFile: Creating file
/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/zoo.cfg
in /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper
with base port 22181
2015-01-28 14:49:31,970 INFO org.apache.giraph.zk.ZooKeeperManager:
generateZooKeeperConfigFile: Make directory of _bspZooKeeper = true
2015-01-28 14:49:31,970 INFO org.apache.giraph.zk.ZooKeeperManager:
generateZooKeeperConfigFile: Delete of zoo.cfg = false
2015-01-28 14:49:31,989 INFO org.apache.giraph.zk.ZooKeeperManager:
onlineZooKeeperServers: Connect attempt 0 of 10 max trying to connect
to fredrik-virtualbox:22181 with poll msecs = 3000
2015-01-28 14:49:32,000 WARN org.apache.giraph.zk.ZooKeeperManager:
onlineZooKeeperServers: Got ConnectException
java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.apache.giraph.zk.ZooKeeperManager.onlineZooKeeperServers(ZooKeeperManager.java:601)
	at org.apache.giraph.graph.GraphTaskManager.startZooKeeperManager(GraphTaskManager.java:433)
	at org.apache.giraph.graph.GraphTaskManager.setup(GraphTaskManager.java:231)
	at org.apache.giraph.graph.GraphMapper.setup(GraphMapper.java:55)
	at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:89)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
	at org.apache.hadoop.mapred.Child.main(Child.java:249)
2015-01-28 14:49:32,018 INFO
org.apache.zookeeper.server.quorum.QuorumPeerConfig: Reading
configuration from:
/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/zoo.cfg
2015-01-28 14:49:32,029 INFO
org.apache.zookeeper.server.DatadirCleanupManager:
autopurge.snapRetainCount set to 3
2015-01-28 14:49:32,029 INFO
org.apache.zookeeper.server.DatadirCleanupManager:
autopurge.purgeInterval set to 0
2015-01-28 14:49:32,029 INFO
org.apache.zookeeper.server.DatadirCleanupManager: Purge task is not
scheduled.
2015-01-28 14:49:32,035 WARN
org.apache.giraph.zk.InProcessZooKeeperRunner: Either no config or no
quorum defined in config, running  in standalone mode
2015-01-28 14:49:32,154 INFO
org.apache.zookeeper.server.quorum.QuorumPeerConfig: Reading
configuration from:
/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/zoo.cfg
2015-01-28 14:49:32,154 INFO
org.apache.zookeeper.server.ZooKeeperServerMain: Starting server
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52
GMT
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:host.name=fredrik-VirtualBox
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:java.version=1.7.0_65
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:java.vendor=Oracle Corporation
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:java.class.path=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars/classes:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work:/usr/local/hadoop/libexec/../conf:/usr/lib/jvm/java-7-openjdk-amd64/lib/tools.jar:/usr/local/hadoop/libexec/..:/usr/local/hadoop/libexec/../hadoop-core-1.2.1.jar:/usr/local/hadoop/libexec/../lib/asm-3.2.jar:/usr/local/hadoop/libexec/../lib/aspectjrt-1.6.11.jar:/usr/local/hadoop/libexec/../lib/aspectjtools-1.6.11.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop/libexec/../lib/commons-cli-1.2.jar:/usr/local/hadoop/libexec/../lib/commons-codec-1.4.jar:/usr/local/hadoop/libexec/../lib/commons-collections-3.2.1.jar:/usr/local/hadoop/libexec/../lib/commons-configuration-1.6.jar:/usr/local/hadoop/libexec/../lib/commons-daemon-1.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-digester-1.8.jar:/usr/local/hadoop/libexec/../lib/commons-el-1.0.jar:/usr/local/hadoop/libexec/../lib/commons-httpclient-3.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-io-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-lang-2.4.jar:/usr/local/hadoop/libexec/../lib/commons-logging-1.1.1.jar:/usr/local/hadoop/libexec/../lib/commons-logging-api-1.0.4.jar:/usr/local/hadoop/libexec/../lib/commons-math-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-net-3.1.jar:/usr/local/hadoop/libexec/../lib/core-3.1.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-capacity-scheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-fairscheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-thriftfs-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop/libexec/../lib/jackson-core-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jasper-compiler-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jasper-runtime-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jdeb-0.8.jar:/usr/local/hadoop/libexec/../lib/jersey-core-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-json-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-server-1.8.jar:/usr/local/hadoop/libexec/../lib/jets3t-0.6.1.jar:/usr/local/hadoop/libexec/../lib/jetty-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jetty-util-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jsch-0.1.42.jar:/usr/local/hadoop/libexec/../lib/junit-4.5.jar:/usr/local/hadoop/libexec/../lib/kfs-0.2.2.jar:/usr/local/hadoop/libexec/../lib/log4j-1.2.15.jar:/usr/local/hadoop/libexec/../lib/mockito-all-1.8.5.jar:/usr/local/hadoop/libexec/../lib/oro-2.0.8.jar:/usr/local/hadoop/libexec/../lib/servlet-api-2.5-20081211.jar:/usr/local/hadoop/libexec/../lib/slf4j-api-1.4.3.jar:/usr/local/hadoop/libexec/../lib/slf4j-log4j12-1.4.3.jar:/usr/local/hadoop/libexec/../lib/xmlenc-0.52.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-api-2.1.jar
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:java.library.path=/usr/local/hadoop/libexec/../lib/native/Linux-amd64-64:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:java.io.tmpdir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work/tmp
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:java.compiler=<NA>
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:os.name=Linux
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:os.arch=amd64
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:os.version=3.16.0-29-generic
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:user.name=fredrik
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:user.home=/home/fredrik
2015-01-28 14:49:32,183 INFO
org.apache.zookeeper.server.ZooKeeperServer: Server
environment:user.dir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work
2015-01-28 14:49:32,215 INFO
org.apache.zookeeper.server.ZooKeeperServer: tickTime set to 6000
2015-01-28 14:49:32,215 INFO
org.apache.zookeeper.server.ZooKeeperServer: minSessionTimeout set to
600000
2015-01-28 14:49:32,216 INFO
org.apache.zookeeper.server.ZooKeeperServer: maxSessionTimeout set to
900000
2015-01-28 14:49:32,266 INFO
org.apache.zookeeper.server.NIOServerCnxnFactory: binding to port
0.0.0.0/0.0.0.0:22181
2015-01-28 14:49:32,337 INFO
org.apache.zookeeper.server.persistence.FileTxnSnapLog: Snapshotting:
0x0 to /app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/work/_bspZooKeeper/version-2/snapshot.0
2015-01-28 14:49:32,414 INFO
org.apache.zookeeper.server.PrepRequestProcessor:
zookeeper.skipACL=="yes", ACL checks will be skipped
2015-01-28 14:49:35,030 INFO org.apache.giraph.zk.ZooKeeperManager:
onlineZooKeeperServers: Connect attempt 1 of 10 max trying to connect
to fredrik-virtualbox:22181 with poll msecs = 3000
2015-01-28 14:49:35,030 INFO org.apache.giraph.zk.ZooKeeperManager:
onlineZooKeeperServers: Connected to
fredrik-VirtualBox/192.168.56.10:22181!
2015-01-28 14:49:35,031 INFO org.apache.giraph.zk.ZooKeeperManager:
onlineZooKeeperServers: Creating my filestamp
_bsp/_defaultZkManagerDir/job_201501281117_0016/_zkServer/fredrik-virtualbox
0
2015-01-28 14:49:35,031 INFO
org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket
connection from /192.168.56.10:34203
2015-01-28 14:49:35,798 INFO
org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket
connection from /192.168.56.10:34204
2015-01-28 14:49:35,804 INFO
org.apache.zookeeper.server.ZooKeeperServer: Client attempting to
establish new session at /192.168.56.10:34204
2015-01-28 14:49:35,807 INFO
org.apache.zookeeper.server.persistence.FileTxnLog: Creating new log
file: log.1
2015-01-28 14:49:35,825 INFO
org.apache.zookeeper.server.ZooKeeperServer: Established session
0x14b30cddb450000 with negotiated timeout 600000 for client
/192.168.56.10:34204
2015-01-28 14:49:35,989 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000
type:create cxid:0x1 zxid:0x2 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode = NoNode
for /_hadoopBsp/job_201501281117_0016
2015-01-28 14:49:36,012 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000
type:create cxid:0x8 zxid:0x7 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0
Error:KeeperErrorCode = NoNode for
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0
2015-01-28 14:49:36,013 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000
type:create cxid:0x9 zxid:0x8 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp Error:KeeperErrorCode = NodeExists for /_hadoopBsp
2015-01-28 14:49:36,013 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000
type:create cxid:0xa zxid:0x9 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode =
NodeExists for /_hadoopBsp/job_201501281117_0016
2015-01-28 14:49:36,014 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000
type:create cxid:0xb zxid:0xa txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir
2015-01-28 14:49:36,023 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000
type:create cxid:0xf zxid:0xd txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
Error:KeeperErrorCode = NoNode for
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
2015-01-28 14:49:36,025 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000
type:create cxid:0x10 zxid:0xe txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp Error:KeeperErrorCode = NodeExists for /_hadoopBsp
2015-01-28 14:49:36,027 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000
type:create cxid:0x11 zxid:0xf txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode =
NodeExists for /_hadoopBsp/job_201501281117_0016
2015-01-28 14:49:36,028 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000
type:create cxid:0x12 zxid:0x10 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir
2015-01-28 14:49:36,029 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000
type:create cxid:0x13 zxid:0x11 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0
2015-01-28 14:49:36,030 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000
type:create cxid:0x14 zxid:0x12 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir
2015-01-28 14:49:36,415 WARN
org.apache.zookeeper.server.NIOServerCnxn: caught end of stream
exception
EndOfStreamException: Unable to read additional data from client
sessionid 0x0, likely client has closed socket
	at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:220)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
	at java.lang.Thread.run(Thread.java:745)
2015-01-28 14:49:36,415 INFO
org.apache.zookeeper.server.NIOServerCnxn: Closed socket connection
for client /192.168.56.10:34203 (no session established for client)
2015-01-28 14:49:37,361 INFO org.apache.giraph.graph.GraphTaskManager:
setup: Chosen to run ZooKeeper...
2015-01-28 14:49:37,362 INFO org.apache.giraph.graph.GraphTaskManager:
setup: Starting up BspServiceMaster (master thread)...
2015-01-28 14:49:37,374 INFO org.apache.giraph.bsp.BspService:
BspService: Path to create to halt is
/_hadoopBsp/job_201501281117_0016/_haltComputation
2015-01-28 14:49:37,374 INFO org.apache.giraph.bsp.BspService:
BspService: Connecting to ZooKeeper with job job_201501281117_0016, 0
on fredrik-virtualbox:22181
2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client
environment:zookeeper.version=3.4.5-1392090, built on 09/30/2012 17:52
GMT
2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client
environment:host.name=fredrik-VirtualBox
2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.version=1.7.0_65
2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.vendor=Oracle Corporation
2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.home=/usr/lib/jvm/java-7-openjdk-amd64/jre
2015-01-28 14:49:37,378 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.class.path=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars/classes:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/jars:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work:/usr/local/hadoop/libexec/../conf:/usr/lib/jvm/java-7-openjdk-amd64/lib/tools.jar:/usr/local/hadoop/libexec/..:/usr/local/hadoop/libexec/../hadoop-core-1.2.1.jar:/usr/local/hadoop/libexec/../lib/asm-3.2.jar:/usr/local/hadoop/libexec/../lib/aspectjrt-1.6.11.jar:/usr/local/hadoop/libexec/../lib/aspectjtools-1.6.11.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-1.7.0.jar:/usr/local/hadoop/libexec/../lib/commons-beanutils-core-1.8.0.jar:/usr/local/hadoop/libexec/../lib/commons-cli-1.2.jar:/usr/local/hadoop/libexec/../lib/commons-codec-1.4.jar:/usr/local/hadoop/libexec/../lib/commons-collections-3.2.1.jar:/usr/local/hadoop/libexec/../lib/commons-configuration-1.6.jar:/usr/local/hadoop/libexec/../lib/commons-daemon-1.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-digester-1.8.jar:/usr/local/hadoop/libexec/../lib/commons-el-1.0.jar:/usr/local/hadoop/libexec/../lib/commons-httpclient-3.0.1.jar:/usr/local/hadoop/libexec/../lib/commons-io-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-lang-2.4.jar:/usr/local/hadoop/libexec/../lib/commons-logging-1.1.1.jar:/usr/local/hadoop/libexec/../lib/commons-logging-api-1.0.4.jar:/usr/local/hadoop/libexec/../lib/commons-math-2.1.jar:/usr/local/hadoop/libexec/../lib/commons-net-3.1.jar:/usr/local/hadoop/libexec/../lib/core-3.1.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-capacity-scheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-fairscheduler-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hadoop-thriftfs-1.2.1.jar:/usr/local/hadoop/libexec/../lib/hsqldb-1.8.0.10.jar:/usr/local/hadoop/libexec/../lib/jackson-core-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jackson-mapper-asl-1.8.8.jar:/usr/local/hadoop/libexec/../lib/jasper-compiler-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jasper-runtime-5.5.12.jar:/usr/local/hadoop/libexec/../lib/jdeb-0.8.jar:/usr/local/hadoop/libexec/../lib/jersey-core-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-json-1.8.jar:/usr/local/hadoop/libexec/../lib/jersey-server-1.8.jar:/usr/local/hadoop/libexec/../lib/jets3t-0.6.1.jar:/usr/local/hadoop/libexec/../lib/jetty-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jetty-util-6.1.26.jar:/usr/local/hadoop/libexec/../lib/jsch-0.1.42.jar:/usr/local/hadoop/libexec/../lib/junit-4.5.jar:/usr/local/hadoop/libexec/../lib/kfs-0.2.2.jar:/usr/local/hadoop/libexec/../lib/log4j-1.2.15.jar:/usr/local/hadoop/libexec/../lib/mockito-all-1.8.5.jar:/usr/local/hadoop/libexec/../lib/oro-2.0.8.jar:/usr/local/hadoop/libexec/../lib/servlet-api-2.5-20081211.jar:/usr/local/hadoop/libexec/../lib/slf4j-api-1.4.3.jar:/usr/local/hadoop/libexec/../lib/slf4j-log4j12-1.4.3.jar:/usr/local/hadoop/libexec/../lib/xmlenc-0.52.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-2.1.jar:/usr/local/hadoop/libexec/../lib/jsp-2.1/jsp-api-2.1.jar
2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.library.path=/usr/local/hadoop/libexec/../lib/native/Linux-amd64-64:/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work
2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.io.tmpdir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work/tmp
2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client
environment:java.compiler=<NA>
2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client
environment:os.name=Linux
2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client
environment:os.arch=amd64
2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client
environment:os.version=3.16.0-29-generic
2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client
environment:user.name=fredrik
2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client
environment:user.home=/home/fredrik
2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper: Client
environment:user.dir=/app/hadoop/tmp/mapred/local/taskTracker/fredrik/jobcache/job_201501281117_0016/attempt_201501281117_0016_m_000000_0/work
2015-01-28 14:49:37,379 INFO org.apache.zookeeper.ZooKeeper:
Initiating client connection, connectString=fredrik-virtualbox:22181
sessionTimeout=60000
watcher=org.apache.giraph.master.BspServiceMaster@f9aac17
2015-01-28 14:49:37,393 INFO org.apache.zookeeper.ClientCnxn: Opening
socket connection to server fredrik-VirtualBox/192.168.56.10:22181.
Will not attempt to authenticate using SASL (unknown error)
2015-01-28 14:49:37,393 INFO
org.apache.zookeeper.server.NIOServerCnxnFactory: Accepted socket
connection from /192.168.56.10:34205
2015-01-28 14:49:37,394 INFO org.apache.zookeeper.ClientCnxn: Socket
connection established to fredrik-VirtualBox/192.168.56.10:22181,
initiating session
2015-01-28 14:49:37,394 INFO
org.apache.zookeeper.server.ZooKeeperServer: Client attempting to
establish new session at /192.168.56.10:34205
2015-01-28 14:49:37,395 INFO
org.apache.zookeeper.server.ZooKeeperServer: Established session
0x14b30cddb450001 with negotiated timeout 600000 for client
/192.168.56.10:34205
2015-01-28 14:49:37,395 INFO org.apache.zookeeper.ClientCnxn: Session
establishment complete on server
fredrik-VirtualBox/192.168.56.10:22181, sessionid = 0x14b30cddb450001,
negotiated timeout = 600000
2015-01-28 14:49:37,396 INFO org.apache.giraph.bsp.BspService:
process: Asynchronous connection complete.
2015-01-28 14:49:37,402 INFO org.apache.giraph.graph.GraphTaskManager:
map: No need to do anything when not a worker
2015-01-28 14:49:37,402 INFO org.apache.giraph.graph.GraphTaskManager:
cleanup: Starting for MASTER_ZOOKEEPER_ONLY
2015-01-28 14:49:37,404 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001
type:create cxid:0x1 zxid:0x17 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_masterElectionDir
Error:KeeperErrorCode = NoNode for
/_hadoopBsp/job_201501281117_0016/_masterElectionDir
2015-01-28 14:49:37,405 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001
type:create cxid:0x2 zxid:0x18 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp Error:KeeperErrorCode = NodeExists for /_hadoopBsp
2015-01-28 14:49:37,406 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001
type:create cxid:0x3 zxid:0x19 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode =
NodeExists for /_hadoopBsp/job_201501281117_0016
2015-01-28 14:49:37,409 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001
type:create cxid:0x6 zxid:0x1c txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_masterJobState
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201501281117_0016/_masterJobState
2015-01-28 14:49:37,409 INFO org.apache.giraph.bsp.BspService:
getJobState: Job state already exists
(/_hadoopBsp/job_201501281117_0016/_masterJobState)
2015-01-28 14:49:37,411 INFO
org.apache.giraph.master.BspServiceMaster: becomeMaster: First child
is '/_hadoopBsp/job_201501281117_0016/_masterElectionDir/fredrik-virtualbox_00000000000'
and my bid is '/_hadoopBsp/job_201501281117_0016/_masterElectionDir/fredrik-virtualbox_00000000000'
2015-01-28 14:49:37,414 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001
type:create cxid:0x9 zxid:0x1d txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir
2015-01-28 14:49:37,414 INFO org.apache.giraph.bsp.BspService:
getApplicationAttempt: Node
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir already
exists!
2015-01-28 14:49:37,477 INFO org.apache.giraph.comm.netty.NettyServer:
NettyServer: Using execution group with 8 threads for
requestFrameDecoder.
2015-01-28 14:49:37,537 INFO org.apache.giraph.comm.netty.NettyServer:
start: Started server communication server:
fredrik-VirtualBox/192.168.56.10:30000 with up to 16 threads on bind
attempt 0 with sendBufferSize = 32768 receiveBufferSize = 524288
2015-01-28 14:49:37,545 INFO org.apache.giraph.comm.netty.NettyClient:
NettyClient: Using execution handler with 8 threads after
request-encoder.
2015-01-28 14:49:37,548 INFO
org.apache.giraph.master.BspServiceMaster: becomeMaster: I am now the
master!
2015-01-28 14:49:37,549 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001
type:create cxid:0xb zxid:0x1e txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir
2015-01-28 14:49:37,549 INFO org.apache.giraph.bsp.BspService:
getApplicationAttempt: Node
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir already
exists!
2015-01-28 14:49:37,550 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001
type:create cxid:0xd zxid:0x1f txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
2015-01-28 14:49:37,559 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001
type:create cxid:0x13 zxid:0x21 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
2015-01-28 14:49:37,560 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001
type:create cxid:0x14 zxid:0x22 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir
2015-01-28 14:49:37,568 INFO
org.apache.giraph.io.formats.GiraphFileInputFormat: Total input paths
to process : 1
2015-01-28 14:49:37,580 WARN
org.apache.hadoop.io.compress.snappy.LoadSnappy: Snappy native library
not loaded
2015-01-28 14:49:37,586 INFO
org.apache.giraph.master.BspServiceMaster: generateVertexInputSplits:
Got 1 input splits for 1 input threads
2015-01-28 14:49:37,586 INFO
org.apache.giraph.master.BspServiceMaster: createVertexInputSplits:
Starting to write input split data to zookeeper with 1 threads
2015-01-28 14:49:37,590 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001
type:create cxid:0x18 zxid:0x23 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_vertexInputSplitDir
Error:KeeperErrorCode = NoNode for
/_hadoopBsp/job_201501281117_0016/_vertexInputSplitDir
2015-01-28 14:49:37,591 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001
type:create cxid:0x19 zxid:0x24 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp Error:KeeperErrorCode = NodeExists for /_hadoopBsp
2015-01-28 14:49:37,591 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001
type:create cxid:0x1a zxid:0x25 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016 Error:KeeperErrorCode =
NodeExists for /_hadoopBsp/job_201501281117_0016
2015-01-28 14:49:37,594 INFO
org.apache.giraph.master.BspServiceMaster: createVertexInputSplits:
Done writing input split data to zookeeper
2015-01-28 14:49:37,595 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001
type:create cxid:0x1e zxid:0x29 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerHealthyDir
2015-01-28 14:49:37,596 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450001
type:create cxid:0x1f zxid:0x2a txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201501281117_0016/_applicationAttemptsDir/0/_superstepDir/-1/_workerUnhealthyDir
2015-01-28 14:49:37,605 INFO org.apache.giraph.comm.netty.NettyClient:
Using Netty without authentication.
2015-01-28 14:49:37,614 INFO org.apache.giraph.comm.netty.NettyClient:
connectAllAddresses: Successfully added 1 connections, (1 total
connected) 0 failed, 0 failures total.
2015-01-28 14:49:37,615 INFO
org.apache.giraph.partition.PartitionUtils: computePartitionCount:
Creating 1, default would have been 1 partitions.
2015-01-28 14:49:37,679 INFO
org.apache.giraph.master.BspServiceMaster: barrierOnWorkerList: 0 out
of 1 workers finished on superstep -1 on path
/_hadoopBsp/job_201501281117_0016/_vertexInputSplitDoneDir
2015-01-28 14:49:37,681 INFO org.apache.giraph.comm.netty.NettyServer:
start: Using Netty without authentication.
2015-01-28 14:49:37,683 INFO
org.apache.giraph.master.BspServiceMaster: barrierOnWorkerList:
Waiting on [fredrik-virtualbox_1]
2015-01-28 14:49:37,779 INFO
org.apache.giraph.master.BspServiceMaster: setJobState:
{"_stateKey":"FAILED","_applicationAttemptKey":-1,"_superstepKey":-1}
on superstep -1
2015-01-28 14:49:37,782 INFO
org.apache.zookeeper.server.PrepRequestProcessor: Got user-level
KeeperException when processing sessionid:0x14b30cddb450000
type:create cxid:0x24 zxid:0x32 txntype:-1 reqpath:n/a Error
Path:/_hadoopBsp/job_201501281117_0016/_masterJobState
Error:KeeperErrorCode = NodeExists for
/_hadoopBsp/job_201501281117_0016/_masterJobState
2015-01-28 14:49:37,785 INFO
org.apache.giraph.master.BspServiceMaster: setJobState:
{"_stateKey":"FAILED","_applicationAttemptKey":-1,"_superstepKey":-1}
2015-01-28 14:49:37,785 FATAL
org.apache.giraph.master.BspServiceMaster: failJob: Killing job
job_201501281117_0016
2015-01-28 14:49:37,785 FATAL
org.apache.giraph.master.BspServiceMaster: failJob: exception
java.lang.IllegalStateException: ******* WORKERS
[Worker(hostname=fredrik-virtualbox, MRtaskID=1, port=30001)] FAILED
*******
2015-01-28 14:49:37,830 ERROR org.apache.giraph.master.MasterThread:
masterThread: Master algorithm failed with IllegalStateException
java.lang.IllegalStateException: coordinateVertexInputSplits: Worker
failed during input split (currently not supported)
	at org.apache.giraph.master.BspServiceMaster.coordinateInputSplits(BspServiceMaster.java:1499)
	at org.apache.giraph.master.BspServiceMaster.coordinateSuperstep(BspServiceMaster.java:1666)
	at org.apache.giraph.master.MasterThread.run(MasterThread.java:124)
2015-01-28 14:49:37,831 FATAL
org.apache.giraph.graph.GraphTaskManager: uncaughtException:
OverrideExceptionHandler on thread
org.apache.giraph.master.MasterThread, msg =
java.lang.IllegalStateException: coordinateVertexInputSplits: Worker
failed during input split (currently not supported), exiting...
java.lang.IllegalStateException: java.lang.IllegalStateException:
coordinateVertexInputSplits: Worker failed during input split
(currently not supported)
	at org.apache.giraph.master.MasterThread.run(MasterThread.java:194)
Caused by: java.lang.IllegalStateException:
coordinateVertexInputSplits: Worker failed during input split
(currently not supported)
	at org.apache.giraph.master.BspServiceMaster.coordinateInputSplits(BspServiceMaster.java:1499)
	at org.apache.giraph.master.BspServiceMaster.coordinateSuperstep(BspServiceMaster.java:1666)
	at org.apache.giraph.master.MasterThread.run(MasterThread.java:124)



*Kinds Regards*
Fredrik

Mime
View raw message