giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alessio Arleo <>
Subject Split Master Worker and Zookeeper: "waitUntilAllTasksDone: Still waiting on tasks 1"
Date Wed, 06 May 2015 09:37:19 GMT
Hello everybody

I migrated to Hadoop 2.4.0 from Hadoop 1.2.1 and I am making some tests on my local machine,
pseudo cluster-mode. To make it work, as stated in
I de-activated the “giraph.splitMasterWorker” option. 

The job completed and worked fine, but it could not complete the output committing phase,
not being able to shutoff ZooKeeper. Here’s a fraction of the last part of the log:

015-05-06 11:25:01,741 INFO [main] org.apache.giraph.zk.ZooKeeperManager: waitUntilAllTasksDone:
Still waiting on tasks 1, 
2015-05-06 11:25:04,744 INFO [main] org.apache.giraph.zk.ZooKeeperManager: waitUntilAllTasksDone:
Got 1 and 2 desired (polling period is 3000) on attempt 299
2015-05-06 11:25:04,744 INFO [main] org.apache.giraph.zk.ZooKeeperManager: waitUntilAllTasksDone:
Still waiting on tasks 1, 
2015-05-06 11:25:07,744 ERROR [main] org.apache.giraph.graph.GraphTaskManager: cleanup: Error
offlining zookeeper
java.lang.IllegalStateException: waitUntilAllTasksDone: Tasks did not finish by the maximum
time of 900000 milliseconds
	at org.apache.giraph.zk.ZooKeeperManager.waitUntilAllTasksDone(
	at org.apache.giraph.zk.ZooKeeperManager.offlineZooKeeperServers(
	at org.apache.giraph.graph.GraphTaskManager.cleanup(
	at org.apache.giraph.graph.GraphMapper.cleanup(
	at org.apache.hadoop.mapred.MapTask.runNewMapper(
	at org.apache.hadoop.mapred.YarnChild$
	at Method)
	at org.apache.hadoop.mapred.YarnChild.main(
2015-05-06 11:25:08,555 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1430743932099_0004_m_000000_0
is done. And is in the process of committing
2015-05-06 11:25:08,644 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1430743932099_0004_m_000000_0'
2015-05-06 11:25:08,717 INFO [Thread-13] org.apache.giraph.zk.ZooKeeperManager: run: Shutdown
hook started.
2015-05-06 11:25:08,717 WARN [Thread-13] org.apache.giraph.zk.ZooKeeperManager: onlineZooKeeperServers:
Forced a shutdown hook kill of the ZooKeeper process.

How come am I experiencing this behaviour? Is ZK expecting two tasks to complete but only
one spawned in the first place (because of the split master/worker option)?

I am really sorry but I am a bit confused. Thanks in advance for any help.


Ing. Alessio Arleo

Dottorando in Ingegneria Industriale e dell’Informazione

Dottore Magistrale in Ingegneria Informatica e dell’Automazione
Dottore in Ingegneria Informatica ed Elettronica

Linkedin: <>
Skype: Ing. Alessio Arleo

Tel: +39 075 5853920
Cell: +39 349 0575782


View raw message