ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Magda <dma...@apache.org>
Subject Re: DataFrame support for Ignite queries
Date Wed, 04 Jan 2017 01:07:54 GMT
Great drill down analysis. Thanks, Val!

I think, that this kind of integration between Spark and Ignite will be truly useful if we
support a logical to physical plan conversion executing a query directly over the data cached
in Ignite. Not sure that it worth our efforts to support Ignite just as one more Spark data
source if, eventually, Spark still will prepare RDDs internally by copying data at query execution
time.  

Do you have an understanding on the efforts to support the logical to physical plan conversion?
Do we need to ask Sergi or someone else from the community to investigate this deeper?

—
Denis

> On Jan 2, 2017, at 9:39 PM, Valentin Kulichenko <valentin.kulichenko@gmail.com>
wrote:
> 
> Folks,
> 
> I made some investigation in this area and put my thoughts in the ticket:
> https://issues.apache.org/jira/browse/IGNITE-3084
> 
> Can someone take a look and provide feedback? Feel free to ask questions if
> anything is unclear.
> 
> -Val
> 
> On Tue, Apr 26, 2016 at 5:29 PM, Dmitriy Setrakyan <dsetrakyan@apache.org>
> wrote:
> 
>> Igniters,
>> 
>> Can someone provide some insight on what would it take to support the
>> DataFrame API directly in Spark applications? I am not an expert in this
>> part of the product, so it is very difficult for me to judge.
>> 
>> Thanks,
>> D.
>> 


Mime
View raw message