ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: Apache Ignite as a distributed processing back-ends
Date Wed, 13 Dec 2017 23:42:14 GMT
Those are the valid points. Personally, I would go for the beam-repo way because it will guarantee
that the integration works as expected with every Beam release. This is for instance how Ignite
is integrated with Camel, MyBatis, Zeppelin.

Anyway, here is a ticket. Hope that an Ignite fellow will step in and solve it in the nearest
time:
https://issues.apache.org/jira/browse/IGNITE-7198 <https://issues.apache.org/jira/browse/IGNITE-7198>

—
Denis

> On Dec 12, 2017, at 9:52 PM, Romain Manni-Bucau <rmannibucau@gmail.com> wrote:
> 
> Hosting integrations in impl more than beam makes a lot of sense IMHO while
> you can maintain it and follow beam release cycle. It will enable you to
> evolve faster and optimise/adapt it more accurately. If you dont have the
> resources, beam would fit better and guarantee it works with each release.
> 
> My 2 cts
> 
> Le 13 déc. 2017 00:47, "Lukasz Cwik" <lcwik@google.com> a écrit :
> 
>> Having it inside the Apache Beam repo makes sense and I could see it being
>> a good fit as an IO and as a runner.
>> 
>> On Tue, Dec 12, 2017 at 3:29 PM, Denis Magda <dmagda@apache.org> wrote:
>> 
>>> Hi Romain,
>>> 
>>> Thanks for the reference. Do you prefer to have the Ignite runner in
>>> Beam’s code base?
>>> 
>>> From what I see, the current runners are hosted there:
>>> https://github.com/apache/beam/tree/master/runners
>>> 
>>> As for Ignite community, we would prefer to hold the integration in your
>>> repo.
>>> 
>>> —
>>> Denis
>>> 
>>> On Dec 7, 2017, at 9:54 PM, Romain Manni-Bucau <rmannibucau@gmail.com>
>>> wrote:
>>> 
>>> Hi
>>> 
>>> This sounds awesome to have an Ignite runner which could compete with
>>> hazelcast-jet.
>>> 
>>> The entry point would be https://beam.apache.org/contribute/runner-guide/
>>> IMHO.
>>> 
>>> Being on Ignite cluster also opens a lot of doors - reusing the filesystem
>>> or distributed structures. Very exiting.
>>> 
>>> Le 8 déc. 2017 05:46, "Denis Magda" <dmagda@apache.org> a écrit :
>>> 
>>> Hello Apache Beam fellows!
>>> 
>>> We at Apache Ignite community came across your project and would be happy
>>> to integrate with it.
>>> 
>>> In short, Ignite is a distributed database and computational platform that
>>> has its own map-reduce like component:
>>> https://apacheignite.readme.io/docs/compute-grid
>>> 
>>> The integration will give Beam users an ability to use Ignite as a
>>> distributed processing back-end system and database.
>>> 
>>> How should we proceed? Please share any relevant information.
>>> 
>>> —
>>> Denis
>>> Ignite PMC
>>> 
>>> 
>>> 
>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message