giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peter Morgan <pmorgan...@gmail.com>
Subject Re: Giraph waiting infinitely
Date Tue, 10 Dec 2013 22:23:01 GMT
I'm using both EdgeInputFormat and VertexValueInputFormat.
I've checked each mapper log and most of them just have this:

INFO org.apache.giraph.utils.ProgressableUtils: waitFor:
Future result not ready yet
java.util.concurrent.FutureTask@4281de5c

over and over again - no error anywhere. Even with the logging set to
"debug" there is nothing obvious.
It works with a small amount of data, but then hangs if you give it a
little bit more.

Using Giraph v 1.1, a snapshot from a month ago.


On Tue, Dec 10, 2013 at 7:44 PM, Ameya Vilankar <ameya.vilankar@gmail.com>wrote:

> Which branch of giraph are you using? Also, what InputFormatClass are you
> using? Since, you are using only 10 Mappers, go to jobtracker and the check
> the log for each mapper. There must be an error happening somewhere.
>
>
> On Tue, Dec 10, 2013 at 10:22 AM, Peter Morgan <pmorgan246@gmail.com>wrote:
>
>> I get a similar problem running my own algorithm.
>> It works on a small set of data and around 10 workers, but scaling it up,
>> I get lots of logs like above, and the job sits infinitely.
>> It is happening on the input read for me, usually around half of the
>> workers finish, and half have those logs.
>>
>>
>> On Wed, Dec 4, 2013 at 9:41 AM, Rafal Wojdyla <rav@spotify.com> wrote:
>>
>>> Hi!
>>>
>>> I use Giraph trunk (1.1.0) with java 1.6 and hadoop 2.2.0.x  and I try
>>> to run the example from http://giraph.apache.org/quick_start.html on
>>> the example data from the page - I have the same problem. So I started this
>>> issue: https://issues.apache.org/jira/browse/GIRAPH-806
>>>
>>> Cheers - Rafal
>>>
>>>
>>> On Tue, Dec 3, 2013 at 8:46 PM, Manuel Lagang <manuellagang@gmail.com>wrote:
>>>
>>>> If you expected your job to take a while and you just want to make sure
>>>> the job doesn't fail due to timeouts, you can set the hadoop parameter "
>>>> mapreduce.task.timeout" to a higher value or to 0 for no timeouts at
>>>> all.
>>>>
>>>> If you didn't expect your job to take a while on a single worker, it's
>>>> hard to figure out what went wrong without more context on your code or
>>>> what stage the job is failing (during input loading? which superstep?).
>>>>
>>>>
>>>> On Tue, Dec 3, 2013 at 11:19 AM, Sundara Raghavan Sankaran <
>>>> sundar@crayondata.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> In my giraph job, one of the workers just outputs the same log
>>>>> repeatedly and ultimately the job fails because of it not reporting its
>>>>> status.
>>>>>
>>>>> Here is the log:-
>>>>>
>>>>> 2013-12-03 19:01:01,159 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Future result not
ready
>>>>> yet java.util.concurrent.FutureTask@4281de5c
>>>>> 2013-12-03 19:01:01,159 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for
>>>>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@5c2d8c20
>>>>> 2013-12-03 19:02:01,159 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Future result not
ready
>>>>> yet java.util.concurrent.FutureTask@4281de5c
>>>>> 2013-12-03 19:02:01,159 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for
>>>>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@5c2d8c20
>>>>> 2013-12-03 19:03:01,159 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Future result not
ready
>>>>> yet java.util.concurrent.FutureTask@4281de5c
>>>>> 2013-12-03 19:03:01,160 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for
>>>>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@5c2d8c20
>>>>> 2013-12-03 19:04:01,160 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Future result not
ready
>>>>> yet java.util.concurrent.FutureTask@4281de5c
>>>>> 2013-12-03 19:04:01,160 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for
>>>>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@5c2d8c20
>>>>> 2013-12-03 19:05:01,160 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Future result not
ready
>>>>> yet java.util.concurrent.FutureTask@4281de5c
>>>>> 2013-12-03 19:05:01,160 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for
>>>>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@5c2d8c20
>>>>> 2013-12-03 19:06:01,161 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Future result not
ready
>>>>> yet java.util.concurrent.FutureTask@4281de5c
>>>>> 2013-12-03 19:06:01,161 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for
>>>>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@5c2d8c20
>>>>> 2013-12-03 19:07:01,161 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Future result not
ready
>>>>> yet java.util.concurrent.FutureTask@4281de5c
>>>>> 2013-12-03 19:07:01,161 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for
>>>>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@5c2d8c20
>>>>> 2013-12-03 19:08:01,161 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Future result not
ready
>>>>> yet java.util.concurrent.FutureTask@4281de5c
>>>>> 2013-12-03 19:08:01,162 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for
>>>>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@5c2d8c20
>>>>> 2013-12-03 19:09:01,162 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Future result not
ready
>>>>> yet java.util.concurrent.FutureTask@4281de5c
>>>>> 2013-12-03 19:09:01,162 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for
>>>>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@5c2d8c20
>>>>> 2013-12-03 19:10:01,162 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Future result not
ready
>>>>> yet java.util.concurrent.FutureTask@4281de5c
>>>>> 2013-12-03 19:10:01,162 INFO
>>>>> org.apache.giraph.utils.ProgressableUtils: waitFor: Waiting for
>>>>> org.apache.giraph.utils.ProgressableUtils$FutureWaitable@5c2d8c20
>>>>>
>>>>> What is going on? Is there something I can do so that it doesn't
>>>>> happen.
>>>>>
>>>>>
>>>>> --
>>>>>
>>>>> Thanks,
>>>>>
>>>>> Sundar
>>>>>
>>>>
>>>>
>>>
>>
>

Mime
View raw message