flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gábor Gévay <gga...@gmail.com>
Subject Re: A soft reminder
Date Thu, 30 Jul 2015 17:46:56 GMT
It is working with setSolutionSetUnmanagedMemory(true), thanks!

Gabor



2015-07-30 19:23 GMT+02:00 Andra Lungu <lungu.andra@gmail.com>:
> Could you try adding the following lines to your code?
>
> VertexCentricConfiguration parameters = new VertexCentricConfiguration();
> parameters.setSolutionSetUnmanagedMemory(true);
>
> return graphWithScoredVertices.runVertexCentricIteration(new
> VertexLabelUpdater(delta),
>       new LabelMessenger(), maxIterations, parameters);
>
> Change the functions appropriately in the runVertexCentricIteration call.
> The error seems to be caused by this CompactingHashTable and this parameter
> deactivates its usage. I am very curious what happens in your case. If you
> could tell us the outcome, I'd greately appreciate it.
>
>
> On Thu, Jul 30, 2015 at 7:17 PM, Gábor Gévay <ggab90@gmail.com> wrote:
>
>> Yes, in a VertexCentricIteration with a few million nodes, running
>> locally on my laptop with about 10 GB of memory given to java.
>>
>> Best,
>> Gabor
>>
>>
>>
>> 2015-07-30 18:32 GMT+02:00 Andra Lungu <lungu.andra@gmail.com>:
>> > Hi Gabor,
>> >
>> > Within a delta iteration right?
>> >
>> > On Thu, Jul 30, 2015 at 6:31 PM, Gábor Gévay <ggab90@gmail.com> wrote:
>> >
>> >> Hi,
>> >>
>> >> I have also run into this problem just now. It only happens with much
>> data.
>> >>
>> >> Best regards,
>> >> Gabor
>> >>
>> >>
>> >>
>> >> 2015-07-27 11:35 GMT+02:00 Felix Neutatz <neutatz@googlemail.com>:
>> >> > Hi,
>> >> >
>> >> > I also encountered the EOF exception for a delta iteration with "more
>> >> > data". With less data it works ...
>> >> >
>> >> > Best regards,
>> >> > Felix
>> >> > Am 27.07.2015 10:25 vorm. schrieb "Andra Lungu" <
>> lungu.andra@gmail.com>:
>> >> >
>> >> >> Hi Stephan,
>> >> >>
>> >> >> I tried to debug a bit around the EOF Exception. It seems that
I am
>> >> pretty
>> >> >> useless on my own. I have no idea how to fix it. And it occurs
in
>> quite
>> >> a
>> >> >> lot of experiments... I believe it's something behind a delta
>> iteration
>> >> >> [this is the common point between my experiments and the others'
>> code].
>> >> >>
>> >> >> Some pointers would be greatly appreciated :)
>> >> >> Thanks!
>> >> >> Andra
>> >> >>
>> >> >> On Mon, Jul 27, 2015 at 9:32 AM, Stephan Ewen <sewen@apache.org>
>> wrote:
>> >> >>
>> >> >> > Hey!
>> >> >> >
>> >> >> > It is pretty much as you said: No one had the time to fix
this so
>> far.
>> >> >> They
>> >> >> > are on the list, though,
>> >> >> >
>> >> >> > Do you want to pick up one of them?
>> >> >> >
>> >> >> > Greetings,
>> >> >> > Stephan
>> >> >> >
>> >> >> >
>> >> >> > On Fri, Jul 24, 2015 at 6:56 PM, Andra Lungu <
>> lungu.andra@gmail.com>
>> >> >> > wrote:
>> >> >> >
>> >> >> > > Hi guys,
>> >> >> > >
>> >> >> > > I opened these JIRAs a while ago and, unfortunately,
no one had
>> >> time to
>> >> >> > > look at them :|.
>> >> >> > >
>> >> >> > > [1] https://issues.apache.org/jira/browse/FLINK-2361
>> >> >> > > [2] https://issues.apache.org/jira/browse/FLINK-2360
>> >> >> > >
>> >> >> > > [1] could probably be hacked, but [2] is a huge problem
that was
>> >> >> > > encountered by other people outside of Gelly.
>> >> >> > >
>> >> >> > > Thanks for the help!
>> >> >> > > Andra
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>>

Mime
View raw message