phoenix-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-5059) Use the Datasource v2 api in the spark connector
Date Thu, 03 Jan 2019 05:52:00 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-5059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16732689#comment-16732689
] 

ASF GitHub Bot commented on PHOENIX-5059:
-----------------------------------------

Github user twdsilva commented on the issue:

    https://github.com/apache/phoenix/pull/423
  
    The PR adds ```PhoenixDataSource``` which implements the new [DataSourceV2 API](https://issues.apache.org/jira/browse/SPARK-15689).
This will allow further optimizations in the future such as limit and aggregate pushdown,
see [SPARK-22386](https://issues.apache.org/jira/browse/SPARK-22386) 
    
    ```PhoenixSparkIT``` and ```PhoenixSparkITTenantSpecific``` were converted to use this
API. 
    ```PhoenixRelation``` and related classes that implemented the old API were deprecated.
    I also bumped up the spark version to  2.4.0 and set the java source and target version
to 1.8 for the maven-compiler-plugin.
    
    @jmahonin can you please review?



> Use the Datasource v2 api in the spark connector
> ------------------------------------------------
>
>                 Key: PHOENIX-5059
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-5059
>             Project: Phoenix
>          Issue Type: New Feature
>            Reporter: Thomas D'Silva
>            Assignee: Thomas D'Silva
>            Priority: Major
>         Attachments: PHOENIX-5059-wip.patch
>
>
> Switching to the v2 api will enable pushing downing limits/aggregations and more in the
future.
> https://issues.apache.org/jira/browse/SPARK-15689



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message