flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mihail Vieru <vi...@informatik.hu-berlin.de>
Subject Flink hanging between job executions / All Pairs Shortest Paths
Date Wed, 13 May 2015 12:45:18 GMT
Hi all,

I've got a problem when running the attached APSPNaiveJob on a graph 
with just 1000 vertices (local execution; 0.9-SNAPSHOT).
It solves the AllPairsShortestPaths problem the naive way - executing 
SingleSourceShortestPaths n times - and storing the computed distances 
in a distance vector for each vertex.

The problem is that Flink almost comes to a standstill when it reaches 
20th iteration, i.e. computing SSSP with srcVertexId = 20. The net 
runtime is becoming increasingly larger than the total runtime by each 
iteration, Flink hanging between executions.

I didn't have this problem when each vertex didn't contain a distance 
vector, but just one distance value. It ran SSSP 1000 times without any 
issues.

The loop:

/        while (srcVertexId < numOfVertices) {//
//            System.out.println("!!! Executing SSSP for srcVertexId = " 
+ srcVertexId);//
////
//            graph = graph.run(new APSP<Long>(srcVertexId, 
maxIterations));//
//
//            graph.getVertices().print();//
//
////            intermediateResult = env.execute("APSPNaive");//
//            jobRuntime += intermediateResult.getNetRuntime();//
/
//            srcVertexId++;//
/        }/

And the program arguments (first being *srcVertexId* and second 
*numOfVertices* used in the loop):

/0 30 
/home/vieru/dev/flink-experiments/data/social_network.verticeslistwweights-1k2 
/home/vieru/dev/flink-experiments/data/social_network.edgelist-1k 
/home/vieru/dev/flink-experiments/sssp-output-x-higgstwitter 10/

Do you know what could cause this problem?

I would greatly appreciate any help.

Best,
Mihail

Mime
View raw message