flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mihail Vieru <vi...@informatik.hu-berlin.de>
Subject Re: RuntimeException Gelly API: Memory ran out. Compaction failed.
Date Wed, 18 Mar 2015 22:25:36 GMT
noooo waaaaay... that was it!? :))) Big thanks! :)
The result is also correct now.

Cheers,
M.

On 18.03.2015 22:49, Vasiliki Kalavri wrote:
> haha, yes, actually I just confirmed!
> If I flip my args, I get the error you mention in the first e-mail. 
> you're trying to generate a graph giving the edge list as a vertex 
> list and this is a way too big dataset for your memory settings (cmp. 
> ~15m edges vs. the actual 400k).
>
> I hope that clear everything out :-)
>
> Cheers,
> V.
>
> On 18 March 2015 at 23:44, Vasiliki Kalavri <vasilikikalavri@gmail.com 
> <mailto:vasilikikalavri@gmail.com>> wrote:
>
>     Well, one thing I notice is that your vertices and edges args are
>     flipped. Might be the source of error :-)
>
>     On 18 March 2015 at 23:04, Mihail Vieru
>     <vieru@informatik.hu-berlin.de
>     <mailto:vieru@informatik.hu-berlin.de>> wrote:
>
>         I'm also using 0 as sourceID. The exact program arguments:
>
>         0
>         /home/vieru/dev/flink-experiments/data/social_network.edgelist
>         /home/vieru/dev/flink-experiments/data/social_network.verticeslist
>         /home/vieru/dev/flink-experiments/sssp-output-higgstwitter 10
>
>         And yes, I call both methods on the initialized Graph
>         *mappedInput*. I don't understand why the distances are
>         computed correctly for the small graph (also read from files)
>         but not for the larger one.
>         The messages appear to be wrong in the latter case.
>
>
>         On 18.03.2015 21:55, Vasiliki Kalavri wrote:
>>         hmm, I'm starting to run out of ideas...
>>         What's your source ID parameter? I ran mine with 0.
>>         About the result, you call both
>>         createVertexCentricIteration() and
>>         runVertexCentricIteration() on the initialized graph, right?
>>
>>         On 18 March 2015 at 22:33, Mihail Vieru
>>         <vieru@informatik.hu-berlin.de
>>         <mailto:vieru@informatik.hu-berlin.de>> wrote:
>>
>>             Hi Vasia,
>>
>>             yes, I am using the latest master. I just did a pull
>>             again and the problem persists. Perhaps Robert could
>>             confirm as well.
>>
>>             I've set the solution set to unmanaged in SSSPUnweighted
>>             as Stephan proposed and the job finishes. So I am able to
>>             proceed using this workaround.
>>
>>             An odd thing occurs now though. The distances aren't
>>             computed correctly for the SNAP graph and remain the one
>>             set in InitVerticesMapper(). For the small graph in
>>             SSSPDataUnweighted they are OK. I'm currently
>>             investigating this behavior.
>>
>>             Cheers,
>>             Mihail
>>
>>
>>             On 18.03.2015 20:55, Vasiliki Kalavri wrote:
>>>             Hi Mihail,
>>>
>>>             I used your code to generate the vertex file, then gave
>>>             this and the edge list as input to your SSSP
>>>             implementation and still couldn't reproduce the
>>>             exception. I'm using the same local setup as I describe
>>>             above.
>>>             I'm not aware of any recent changes that might be
>>>             relevant, but, just in case, are you using the latest
>>>             master?
>>>
>>>             Cheers,
>>>             V.
>>>
>>>             On 18 March 2015 at 19:21, Mihail Vieru
>>>             <vieru@informatik.hu-berlin.de
>>>             <mailto:vieru@informatik.hu-berlin.de>> wrote:
>>>
>>>                 Hi Vasia,
>>>
>>>                 I have used a simple job (attached) to generate a
>>>                 file which looks like this:
>>>
>>>                 0 0
>>>                 1 1
>>>                 2 2
>>>                 ...
>>>                 456629 456629
>>>                 456630 456630
>>>
>>>                 I need the vertices to be generated from a file for
>>>                 my future work.
>>>
>>>                 Cheers,
>>>                 Mihail
>>>
>>>
>>>
>>>                 On 18.03.2015 17:04, Vasiliki Kalavri wrote:
>>>>                 Hi Mihail, Robert,
>>>>
>>>>                 I've tried reproducing this, but I couldn't.
>>>>                 I'm using the same twitter input graph from SNAP
>>>>                 that you link to and also Scala IDE.
>>>>                 The job finishes without a problem (both the SSSP
>>>>                 example from Gelly and the unweighted version).
>>>>
>>>>                 The only thing I changed to run your version was
>>>>                 creating the graph from the edge set only, i.e.
>>>>                 like this:
>>>>
>>>>                 Graph<Long, Long, NullValue> graph =
>>>>                 Graph.fromDataSet(edges,
>>>>                 new MapFunction<Long, Long>() {
>>>>                 public Long map(Long value) {
>>>>                 return Long.MAX_VALUE;
>>>>                 }
>>>>                 }, env);
>>>>                 Since the twitter input is an edge list, how do you
>>>>                 generate the vertex dataset in your case?
>>>>
>>>>                 Thanks,
>>>>                 -Vasia.
>>>>
>>>>                 On 18 March 2015 at 16:54, Mihail Vieru
>>>>                 <vieru@informatik.hu-berlin.de
>>>>                 <mailto:vieru@informatik.hu-berlin.de>> wrote:
>>>>
>>>>                     Hi,
>>>>
>>>>                     great! Thanks!
>>>>
>>>>                     I really need this bug fixed because I'm laying
>>>>                     the groundwork for my Diplom thesis and I need
>>>>                     to be sure that the Gelly API is reliable and
>>>>                     can handle large datasets as intended.
>>>>
>>>>                     Cheers,
>>>>                     Mihail
>>>>
>>>>
>>>>                     On 18.03.2015 15:40, Robert Waury wrote:
>>>>>                     Hi,
>>>>>
>>>>>                     I managed to reproduce the behavior and as far
>>>>>                     as I can tell it seems to be a problem with
>>>>>                     the memory allocation.
>>>>>
>>>>>                     I have filed a bug report in JIRA to get the
>>>>>                     attention of somebody who knows the runtime
>>>>>                     better than I do.
>>>>>
>>>>>                     https://issues.apache.org/jira/browse/FLINK-1734
>>>>>
>>>>>                     Cheers,
>>>>>                     Robert
>>>>>
>>>>>                     On Tue, Mar 17, 2015 at 3:52 PM, Mihail Vieru
>>>>>                     <vieru@informatik.hu-berlin.de
>>>>>                     <mailto:vieru@informatik.hu-berlin.de>>
wrote:
>>>>>
>>>>>                         Hi Robert,
>>>>>
>>>>>                         thank you for your reply.
>>>>>
>>>>>                         I'm starting the job from the Scala IDE.
>>>>>                         So only one JobManager and one TaskManager
>>>>>                         in the same JVM.
>>>>>                         I've doubled the memory in the eclipse.ini
>>>>>                         settings but I still get the Exception.
>>>>>
>>>>>                         -vmargs
>>>>>                         -Xmx2048m
>>>>>                         -Xms100m
>>>>>                         -XX:MaxPermSize=512m
>>>>>
>>>>>                         Best,
>>>>>                         Mihail
>>>>>
>>>>>
>>>>>                         On 17.03.2015 10:11, Robert Waury wrote:
>>>>>>                         Hi,
>>>>>>
>>>>>>                         can you tell me how much memory your job
>>>>>>                         has and how many workers you are running?
>>>>>>
>>>>>>                         From the trace it seems the internal hash
>>>>>>                         table allocated only 7 MB for the graph
>>>>>>                         data and therefore runs out of memory
>>>>>>                         pretty quickly.
>>>>>>
>>>>>>                         Skewed data could also be an issue but
>>>>>>                         with a minimum of 5 pages and a maximum
>>>>>>                         of 8 it seems to be distributed fairly
>>>>>>                         even to the different partitions.
>>>>>>
>>>>>>                         Cheers,
>>>>>>                         Robert
>>>>>>
>>>>>>                         On Tue, Mar 17, 2015 at 1:25 AM, Mihail
>>>>>>                         Vieru <vieru@informatik.hu-berlin.de
>>>>>>                         <mailto:vieru@informatik.hu-berlin.de>>
>>>>>>                         wrote:
>>>>>>
>>>>>>                             And the correct SSSPUnweighted attached.
>>>>>>
>>>>>>
>>>>>>                             On 17.03.2015 01:23, Mihail Vieru wrote:
>>>>>>
>>>>>>                                 Hi,
>>>>>>
>>>>>>                                 I'm getting the following
>>>>>>                                 RuntimeException for an
>>>>>>                                 adaptation of the
>>>>>>                                 SingleSourceShortestPaths example
>>>>>>                                 using the Gelly API (see
>>>>>>                                 attachment). It's been adapted
>>>>>>                                 for unweighted graphs having
>>>>>>                                 vertices with Long values.
>>>>>>
>>>>>>                                 As an input graph I'm using the
>>>>>>                                 social network graph (~200MB
>>>>>>                                 unpacked) from here:
>>>>>>                                 https://snap.stanford.edu/data/higgs-twitter.html
>>>>>>
>>>>>>                                 For the small SSSPDataUnweighted
>>>>>>                                 graph (also attached) it
>>>>>>                                 terminates and computes the
>>>>>>                                 distances correctly.
>>>>>>
>>>>>>
>>>>>>                                 03/16/2015 17:18:23
>>>>>>                                 IterationHead(WorksetIteration
>>>>>>                                 (Vertex-centric iteration
>>>>>>                                 (org.apache.flink.graph.library.SingleSourceShortestPathsUnweighted$VertexDistanceUpdater@dca6fe4
>>>>>>                                 |
>>>>>>                                 org.apache.flink.graph.library.SingleSourceShortestPathsUnweighted$MinDistanceMessenger@6577e8ce)))(2/4)
>>>>>>                                 switched to FAILED
>>>>>>                                 java.lang.RuntimeException:
>>>>>>                                 Memory ran out. Compaction
>>>>>>                                 failed. numPartitions: 32
>>>>>>                                 minPartition: 5 maxPartition: 8
>>>>>>                                 number of overflow segments: 176
>>>>>>                                 bucketSize: 217 Overall memory:
>>>>>>                                 20316160 Partition memory:
>>>>>>                                 7208960 Message: Index: 8, Size:
7
>>>>>>                                     at
>>>>>>                                 org.apache.flink.runtime.operators.hash.CompactingHashTable.insert(CompactingHashTable.java:390)
>>>>>>                                     at
>>>>>>                                 org.apache.flink.runtime.operators.hash.CompactingHashTable.buildTable(CompactingHashTable.java:337)
>>>>>>                                     at
>>>>>>                                 org.apache.flink.runtime.iterative.task.IterationHeadPactTask.readInitialSolutionSet(IterationHeadPactTask.java:216)
>>>>>>                                     at
>>>>>>                                 org.apache.flink.runtime.iterative.task.IterationHeadPactTask.run(IterationHeadPactTask.java:278)
>>>>>>                                     at
>>>>>>                                 org.apache.flink.runtime.operators.RegularPactTask.invoke(RegularPactTask.java:362)
>>>>>>                                     at
>>>>>>                                 org.apache.flink.runtime.execution.RuntimeEnvironment.run(RuntimeEnvironment.java:205)
>>>>>>                                     at
>>>>>>                                 java.lang.Thread.run(Thread.java:745)
>>>>>>
>>>>>>
>>>>>>                                 Best,
>>>>>>                                 Mihail
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
>
>
>


Mime
View raw message