spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Takeshi Yamamuro (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-15816) SQL server based on Postgres protocol
Date Fri, 19 Aug 2016 08:56:21 GMT

    [ https://issues.apache.org/jira/browse/SPARK-15816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15427848#comment-15427848
] 

Takeshi Yamamuro commented on SPARK-15816:
------------------------------------------

About 1.
Yea, we can support them in there; nested structs and maps are handled as json-fomatted strings.
Array formats also are handled as strings like '{x,x,x}'.
See tests for struct/map: https://github.com/apache/spark/compare/master...maropu:SPARK-15816#diff-8965860d928dfaf214eaf6a3f724b961R339
array: https://github.com/apache/spark/compare/master...maropu:SPARK-15816#diff-8965860d928dfaf214eaf6a3f724b961R234

About 2.
ISTM it is not impossible though, what's the purpose to support a https mode?

> SQL server based on Postgres protocol
> -------------------------------------
>
>                 Key: SPARK-15816
>                 URL: https://issues.apache.org/jira/browse/SPARK-15816
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>            Reporter: Reynold Xin
>         Attachments: New_SQL_Server_for_Spark.pdf
>
>
> At Spark Summit today this idea came up from a discussion: it would be great to investigate
the possibility of implementing a new SQL server using Postgres' protocol, in lieu of Hive
ThriftServer 2. I'm creating this ticket to track this idea, in case others have feedback.
> This server can have a simpler architecture, and allows users to leverage a wide range
of tools that are already available for Postgres (and many commercial database systems based
on Postgres).
> Some of the problems we'd need to figure out are:
> 1. What is the Postgres protocol? Is there an official documentation for it?
> 2. How difficult would it be to implement that protocol in Spark (JVM in particular).
> 3. How does data type mapping work?
> 4. How does system commands work? Would Spark need to support all of Postgres' commands?
> 5. Any restrictions in supporting nested data?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message