spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tor Myklebust (JIRA)" <>
Subject [jira] [Commented] (SPARK-5472) Add support for reading from and writing to a JDBC database
Date Sat, 31 Jan 2015 04:12:34 GMT


Tor Myklebust commented on SPARK-5472:

You appear to understand the issue perfectly.  You have to write the case class mapper, work
out the schema, and register the thing as a temporary table.  Once you've done all that for
one table, you have to do something rather similar for the next table you want to load.  And
all this work requires Scala coding rather than a short SQL query.  This is more complexity
for the user than the problem really deserves.

> Add support for reading from and writing to a JDBC database
> -----------------------------------------------------------
>                 Key: SPARK-5472
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: SQL
>            Reporter: Tor Myklebust
>            Assignee: Tor Myklebust
>            Priority: Blocker
> It would be nice to be able to make a table in a JDBC database appear as a table in Spark
SQL.  This would let users, for instance, perform a JOIN between a DataFrame in Spark SQL
with a table in a Postgres database.
> It might also be nice to be able to go the other direction -- save a DataFrame to a database
-- for instance in an ETL job.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message