giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Armando Miraglia <a.mirag...@student.vu.nl>
Subject Re: GIRAPH-825 and GIRAPH-840
Date Wed, 12 Feb 2014 11:53:18 GMT

btw: I as also thinking to update the documentation page on the Giraph
website to better explain the sticky partition logic. What do  you
think?

Cheers,
Armando


On Wed, Feb 12, 2014 at 12:50:25PM +0100, Armando Miraglia wrote:
> Indeed, yesterday I was fixing a couple of things and I think I missed a
> case that I have to exclude. Sorry for this, I have fever at the momento
> so it could be that yesterday I was under the effect of the fever :D
> 
> I checked that the tests were passing but I think a missed something.
> 
> I'll come back to you very soon
> 
> On Wed, Feb 12, 2014 at 10:26:12AM +0100, Claudio Martella wrote:
> > the problem is that you're running with more threads than in-memory
> > partitions. increase the number of partitions in memory to be at least the
> > number of threads. i have no time right now to check the latest code, but
> > you should not set the number of stickypartitions by hand.
> > 
> > 
> > On Wed, Feb 12, 2014 at 10:03 AM, Sebastian Schelter <ssc@apache.org> wrote:
> > 
> > > I ran a first test with the new DiskBackedPartitionStore and it didn't
> > > work for me unfortunately. The job never leaves the input phase (superstep
> > > -1). I sshd onto one of the workers and it seems to wait forever on
> > > DiskBackedPartitionStore.getOrCreatePartition:
> > >
> > >    java.lang.Thread.State: BLOCKED (on object monitor)
> > >         at org.apache.giraph.partition.DiskBackedPartitionStore.
> > > getOrCreatePartition(DiskBackedPartitionStore.java:226)
> > >         - waiting to lock <0x00000000aeb757c8> (a
> > > org.apache.giraph.partition.DiskBackedPartitionStore$MetaPartition)
> > >
> > > Here are the custom arguments for my run, let me know if I should do
> > > another run with a different config.
> > >
> > > giraph.oneToAllMsgSending=true
> > > giraph.isStaticGraph=true
> > > giraph.numComputeThreads=15
> > > giraph.numInputThreads=15
> > > giraph.numOutputThreads=15
> > > giraph.maxNumberOfSupersteps=30
> > > giraph.useOutOfCoreGraph=true
> > > giraph.stickyPartitions=5
> > >
> > > I also ran the job without using oneToAllMsgSending and saw the same
> > > behavior.
> > >
> > > Best,
> > > Sebastian
> > >
> > >
> > >
> > > On 02/12/2014 12:44 AM, Claudio Martella wrote:
> > >
> > >> please give it a test. i've been working on this with armando. i'll give
a
> > >> review, but we have been testing it for while. we'd really appreciate if
> > >> somebody else could run some additional tests as well. thanks!
> > >>
> > >>
> > >> On Wed, Feb 12, 2014 at 12:39 AM, Sebastian Schelter <ssc@apache.org>
> > >> wrote:
> > >>
> > >>  I'll test the patch from GIRAPH-825 this week.
> > >>>
> > >>>
> > >>> On 02/12/2014 12:10 AM, Roman Shaposhnik wrote:
> > >>>
> > >>>  Hi!
> > >>>>
> > >>>> Given how big the diff here are:
> > >>>>       https://issues.apache.org/jira/browse/GIRAPH-825
> > >>>>       https://issues.apache.org/jira/browse/GIRAPH-840
> > >>>> I am wondering whether it is realistic
> > >>>> to have them in 1.1.0.
> > >>>>
> > >>>> Would appreciate folks chiming in.
> > >>>>
> > >>>> Thanks,
> > >>>> Roman.
> > >>>>
> > >>>>
> > >>>>
> > >>>
> > >>
> > >>
> > >
> > 
> > 
> > -- 
> >    Claudio Martella

Mime
View raw message