Hi,
When I examine the logs of each worker, I found that some time is spent on
waiting for other tasks:
INFO org.apache.giraph.utils.TaskIdsPermitsBarrier: waitForRequiredPermits:
Waiting for 7 more tasks to send their aggregator data, task ids: [3, 6,
13, 17, 18, 26, 29]
And this seems to me the barrier in BSP model for every super-step.
Do we have a method to aggregate all these waiting time?
Thanks.
--
*JU Han*
Software Engineer Intern @ KXEN Inc.
UTC - Université de Technologie de Compiègne
* **GI06 - Fouille de Données et Décisionnel*
+33 0619608888
|