ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject [DISCUSS] Flink and Ignite integration
Date Tue, 28 Apr 2015 21:57:53 GMT
Following the lively exchange in Twitter (sic!) I would like to bring together
Ignite and Flink communities to discuss the benefits of the integration and
see where we can start it.

We have this recently opened ticket 
  https://issues.apache.org/jira/browse/IGNITE-813

and Fabian has listed the following points:

 1) data store
 2) parameter server for ML models
 3) Checkpointing streaming op state
 4) continuously updating views from streams

I'd add 
 5) using Ignite IGFS to speed up Flink's access to HDFS data.

I see a lot of interesting correlations between two projects and wonder if
Flink guys can step up with a few thoughts on where Flink can benefit the most
from Ignite's in-memory fabric architecture? Perhaps, it can be used as
in-memory storage where the other components of the stack can quickly access
and work w/ the data w/o a need to dump it back to slow storage?

Thoughts?
  Cos

Mime
View raw message