phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-1815) Use Spark Data Source API in phoenix-spark module
Date Mon, 06 Apr 2015 14:05:12 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-1815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14481237#comment-14481237
] 

ASF GitHub Bot commented on PHOENIX-1815:
-----------------------------------------

GitHub user jmahonin opened a pull request:

    https://github.com/apache/phoenix/pull/63

    PHOENIX-1815 Use Spark Data Source API in phoenix-spark module

    This allows using the SQLContext.load() functionality to create
    a Phoenix data frame, which also supports push-down on column
    or predicate filtering from Spark SQL.
    
    As well, DataFrame.save() is supported for persisting DataFrames
    back to Phoenix.
    
    This may work with Spark's standalone SQL server mode, but it
    hasn't been tested.
    
    ref:
    https://spark.apache.org/docs/latest/sql-programming-guide.html#data-sources
    
    https://databricks.com/blog/2015/01/09/spark-sql-data-sources-api-unified-data-access-for-the-spark-platform.html

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/FileTrek/phoenix PHOENIX-1815

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/phoenix/pull/63.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #63
    
----
commit c62178a1016a7885bd2c082fd1380c9a3023ca34
Author: Josh Mahonin <jmahonin@gmail.com>
Date:   2015-03-25T19:40:10Z

    PHOENIX-1815 Use Spark Data Source API in phoenix-spark module
    
    This allows using the SQLContext.load() functionality to create
    a Phoenix data frame, which also supports push-down on column
    or predicate filtering from Spark SQL.
    
    As well, DataFrame.save() is supported for persisting DataFrames
    back to Phoenix.
    
    This may work with Spark's standalone SQL server mode, but it
    hasn't been tested.
    
    ref:
    https://spark.apache.org/docs/latest/sql-programming-guide.html#data-sources
    
    https://databricks.com/blog/2015/01/09/spark-sql-data-sources-api-unified-data-access-for-the-spark-platform.html

----


> Use Spark Data Source API in phoenix-spark module
> -------------------------------------------------
>
>                 Key: PHOENIX-1815
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-1815
>             Project: Phoenix
>          Issue Type: New Feature
>            Reporter: Josh Mahonin
>
> Spark 1.3.0 introduces a new 'Data Source' API to standardize load and save methods for
different types of data sources.
> The phoenix-spark module should implement the same API for use as a pluggable data store
in Spark.
> ref:
> https://spark.apache.org/docs/latest/sql-programming-guide.html#data-sources
>     
> https://databricks.com/blog/2015/01/09/spark-sql-data-sources-api-unified-data-access-for-the-spark-platform.html



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message