giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: GIRAPH-825 and GIRAPH-840
Date Wed, 12 Feb 2014 09:26:12 GMT
the problem is that you're running with more threads than in-memory
partitions. increase the number of partitions in memory to be at least the
number of threads. i have no time right now to check the latest code, but
you should not set the number of stickypartitions by hand.


On Wed, Feb 12, 2014 at 10:03 AM, Sebastian Schelter <ssc@apache.org> wrote:

> I ran a first test with the new DiskBackedPartitionStore and it didn't
> work for me unfortunately. The job never leaves the input phase (superstep
> -1). I sshd onto one of the workers and it seems to wait forever on
> DiskBackedPartitionStore.getOrCreatePartition:
>
>    java.lang.Thread.State: BLOCKED (on object monitor)
>         at org.apache.giraph.partition.DiskBackedPartitionStore.
> getOrCreatePartition(DiskBackedPartitionStore.java:226)
>         - waiting to lock <0x00000000aeb757c8> (a
> org.apache.giraph.partition.DiskBackedPartitionStore$MetaPartition)
>
> Here are the custom arguments for my run, let me know if I should do
> another run with a different config.
>
> giraph.oneToAllMsgSending=true
> giraph.isStaticGraph=true
> giraph.numComputeThreads=15
> giraph.numInputThreads=15
> giraph.numOutputThreads=15
> giraph.maxNumberOfSupersteps=30
> giraph.useOutOfCoreGraph=true
> giraph.stickyPartitions=5
>
> I also ran the job without using oneToAllMsgSending and saw the same
> behavior.
>
> Best,
> Sebastian
>
>
>
> On 02/12/2014 12:44 AM, Claudio Martella wrote:
>
>> please give it a test. i've been working on this with armando. i'll give a
>> review, but we have been testing it for while. we'd really appreciate if
>> somebody else could run some additional tests as well. thanks!
>>
>>
>> On Wed, Feb 12, 2014 at 12:39 AM, Sebastian Schelter <ssc@apache.org>
>> wrote:
>>
>>  I'll test the patch from GIRAPH-825 this week.
>>>
>>>
>>> On 02/12/2014 12:10 AM, Roman Shaposhnik wrote:
>>>
>>>  Hi!
>>>>
>>>> Given how big the diff here are:
>>>>       https://issues.apache.org/jira/browse/GIRAPH-825
>>>>       https://issues.apache.org/jira/browse/GIRAPH-840
>>>> I am wondering whether it is realistic
>>>> to have them in 1.1.0.
>>>>
>>>> Would appreciate folks chiming in.
>>>>
>>>> Thanks,
>>>> Roman.
>>>>
>>>>
>>>>
>>>
>>
>>
>


-- 
   Claudio Martella

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message