flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vasiliki Kalavri <vasilikikala...@gmail.com>
Subject Re: Retrieving a single element from a DataSet
Date Sat, 05 Nov 2016 18:53:43 GMT
Hi all,

@Wouter: I'm not sure I completely understand what you want to do, but
would broadcast variables [1] help?

@all: All-pairs-shortest-paths and betweenness centrality are very
challenging algorithms to implement efficiently in a distributed way. APSP
requires each vertex to store distances (or paths) for every other vertex
in the graph. AFAIK there's no scalable distributed algorithm to compute
these metrics. The solutions I'm aware of are (1) approximations and
sketches (e.g. spanners), (2) replicating the whole graph to several nodes
and compute paths in parallel, and (3) shared-memory implementations that
exploit multi-core parallelism. How are you planning to implement these in
Gelly?

Cheers,
-Vasia.

[1]:
https://ci.apache.org/projects/flink/flink-docs-release-1.1/apis/batch/index.html#broadcast-variables

On 4 November 2016 at 12:57, Greg Hogan <code@greghogan.com> wrote:

> The tickets are in Flink's Jira:
>   https://issues.apache.org/jira/browse/FLINK-4965
>   https://issues.apache.org/jira/browse/FLINK-4966
>
> Are you looking to process temporal graphs with the DataStream API?
>
> On Fri, Nov 4, 2016 at 5:52 AM, otherwise777 <wouter@onzichtbaar.net>
> wrote:
>
>> Cool, thnx for that,
>>
>> I tried searching for it in teh github but couldn't find it, do you have
>> the
>> url by any chance?
>> I'm going to try to implement such an algorithm for temporal graphs
>>
>>
>>
>> --
>> View this message in context: http://apache-flink-user-maili
>> ng-list-archive.2336050.n4.nabble.com/Retrieving-a-single
>> -element-from-a-DataSet-tp9731p9894.html
>> Sent from the Apache Flink User Mailing List archive. mailing list
>> archive at Nabble.com.
>>
>
>

Mime
View raw message