flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vasiliki Kalavri <vasilikikala...@gmail.com>
Subject Re: Apache Tinkerpop & Geode Integration?
Date Wed, 09 Dec 2015 19:40:52 GMT
Hello squirrels,

I have been discussing with the Apache Tinkerpop [1] community regarding an
integration with Flink/Gelly.
You can read our discussion in [2].

Tinkerpop has a graph traversal machine called Gremlin, which supports many
high-level graph processing languages and runs on top of different systems
(e.g. Giraph, Spark, Graph DBs). You can read more in this great blog post
[3].

The idea is to provide a FlinkGraphComputer implementation, which will add
Gremlin support to Flink.

I believe Tinkerpop is a great project and I would love to see an
integration with Gelly.
Before we move forward, I would like your input!
To me, it seems that this addition would nicely fit in flink-contrib, where
we also have connectors to other projects.
If you agree, I will go ahead and open a JIRA about it.

Thank you!
-Vasia.

[1]: https://tinkerpop.incubator.apache.org/
[2]:
https://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/201511.mbox/%3CCANva_A390L7g169r8Sn+ej1-yfKBUdLnd4Td6ATwnP0uzA--gA@mail.gmail.com%3E
[3]:
http://www.datastax.com/dev/blog/the-benefits-of-the-gremlin-graph-traversal-machine

On 25 November 2015 at 16:54, Vasiliki Kalavri <vasilikikalavri@gmail.com>
wrote:

> Hi James,
>
> I've just subscribed to the Tinkerpop dev mailing list. Could you please
> send a reply to the thread, so then I can reply to it?
> I'm not sure how I can reply to the thread otherwise...
> I also saw that there is a grafos.ml project thread. I could also provide
> some input there :)
>
> Thanks!
> -Vasia.
>
> On 25 November 2015 at 15:09, James Thornton <james.thornton@gmail.com>
> wrote:
>
>> Hi Vasia -
>>
>> Yes, a FlinkGraphComputer should be a straight-forward first step. Also,
>> on
>> the Apache Tinkerpop dev mailing list, Marko thought it might be cool if
>> there was a "Graph API" similar to the "Table API" -- hooking in Gremlin
>> to
>> Flink's fluent API would give Flink users a full graph query language.
>>
>> Stephen Mallette is a TinkerPop core contributor, and he has already
>> started working on a FlinkGraphComputer. There is a Flink/Tinkerpop thread
>> on the TinkerPop dev list -- it would be great to have you part of the
>> conversation there too as we work on the integration:
>>
>>    http://mail-archives.apache.org/mod_mbox/incubator-tinkerpop-dev/
>>
>> Thanks, Vasia.
>>
>> - James
>>
>>
>> On Mon, Nov 23, 2015 at 10:28 AM, Vasiliki Kalavri <
>> vasilikikalavri@gmail.com> wrote:
>>
>> > Hi James,
>> >
>> > thank you for your e-mail and your interest in Flink :)
>> >
>> > I've recently taken a _quick_ look into Apache TinkerPop and I think
>> it'd
>> > be very interesting to integrate with Flink/Gelly.
>> > Are you thinking about something like a Flink GraphComputer, similar to
>> > Giraph and Spark GraphComputer's?
>> > I believe such an integration should be straight-forward to implement.
>> You
>> > can start by looking into Flink iteration operators [1] and Gelly
>> iteration
>> > abstractions [2].
>> >
>> > Regarding Apache Geode, I'm not familiar with project, but I'll try to
>> take
>> > a look in the following days!
>> >
>> > Cheers,
>> > -Vasia.
>> >
>> >
>> > [1]:
>> >
>> >
>> https://ci.apache.org/projects/flink/flink-docs-master/apis/programming_guide.html#iteration-operators
>> > [2]:
>> >
>> >
>> https://ci.apache.org/projects/flink/flink-docs-master/libs/gelly_guide.html#iterative-graph-processing
>> >
>> >
>> > On 20 November 2015 at 08:32, James Thornton <james.thornton@gmail.com>
>> > wrote:
>> >
>> > > Hi -
>> > >
>> > > This is James Thornton (espeed) from the Apache Tinkerpop project (
>> > > http://tinkerpop.incubator.apache.org/).
>> > >
>> > > The Flink iterators should pair well with Gremlin's Graph Traversal
>> > Machine
>> > > (
>> > >
>> > >
>> >
>> http://www.datastax.com/dev/blog/the-benefits-of-the-gremlin-graph-traversal-machine
>> > > )
>> > > -- it would be good to coordinate on creating an integration.
>> > >
>> > > Also, Apache Geode made a splash today on HN (
>> > > https://news.ycombinator.com/item?id=10596859) -- connecting Geode
>> and
>> > > Flink would be killer. Here's the Geode/Spark connector for
>> refefference:
>> > >
>> > >
>> > >
>> > >
>> >
>> https://github.com/apache/incubator-geode/tree/develop/gemfire-spark-connector
>> > >
>> > > - James
>> > >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message