carbondata-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fei Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CARBONDATA-322) integrate spark 2.x
Date Tue, 18 Oct 2016 07:38:59 GMT

     [ https://issues.apache.org/jira/browse/CARBONDATA-322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Fei Wang updated CARBONDATA-322:
--------------------------------
    Description: 
As spark 2.0 released. there are many nice features such as more efficient parser, vectorized
execution, adaptive execution. 
It is good to integrate with spark 2.x

Another side now spark integration is heavy coupling with spark, we should redesign the spark
integration, it should satisfy flowing requirement:

1. decoupled with spark, integrate according to spark datasource API(V2)
2. This integration should support vectorized carbon reader
3. Supoort write to carbondata from dadatrame
...


  was:
As spark 2.0 released. there are many nice features such as more efficient parser, vectorized
execution, adaptive execution. It is good to integrate with spark 2.x

Another side now spark integration is heavy coupling with spark, we should redesign the spark
integration, it should satisfy flowing requirement:

1. decoupled with spark, integrate according to spark datasource API(V2)
2. This integration should support vectorized carbon reader
3. Supoort write to carbondata from dadatrame
...



> integrate spark 2.x 
> --------------------
>
>                 Key: CARBONDATA-322
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-322
>             Project: CarbonData
>          Issue Type: Bug
>          Components: spark-integration
>    Affects Versions: 0.2.0-incubating
>            Reporter: Fei Wang
>             Fix For: 0.3.0-incubating
>
>
> As spark 2.0 released. there are many nice features such as more efficient parser, vectorized
execution, adaptive execution. 
> It is good to integrate with spark 2.x
> Another side now spark integration is heavy coupling with spark, we should redesign the
spark integration, it should satisfy flowing requirement:
> 1. decoupled with spark, integrate according to spark datasource API(V2)
> 2. This integration should support vectorized carbon reader
> 3. Supoort write to carbondata from dadatrame
> ...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message