spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Marco Colombo (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-16717) Dataframe (jdbc) is missing a way to link and external function to get a connection
Date Tue, 26 Jul 2016 12:14:20 GMT

     [ https://issues.apache.org/jira/browse/SPARK-16717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Marco Colombo updated SPARK-16717:
----------------------------------
    Description: 
In JdbcRRD it was possible to use a function to get a JDBC connection. This allow an external
handling of the connections while now this is no more possible with dataframes. 
Please consider an addition to Dataframes for using an externally provided connectionFactory
(such as a connection pool) in order to make data loading more efficient, avoiding connection
close/recreation. Connections should be taken from provided function and returned to a second
function whenever no more used by the RRD. So this will make jdbc handling more efficient.

I.e. extending DataFrame class with a method like jdbc(Function0<java.sql.Connection>
getConnection, Function0 releaseConnection(java.sql.Connection))


  was:
In JdbcRRD it was possible to use a function to get a JDBC connection. This allow an external
handling of the connections while now this is no more possible with dataframes. 
Please consider an addition to Dataframes for using an externally provided connectionFactory
(such as a connection pool) in order to make data loading more efficient, avoiding connection
close/recreation. 


> Dataframe (jdbc) is missing a way to link and external function to get a connection
> -----------------------------------------------------------------------------------
>
>                 Key: SPARK-16717
>                 URL: https://issues.apache.org/jira/browse/SPARK-16717
>             Project: Spark
>          Issue Type: Improvement
>    Affects Versions: 1.6.2
>            Reporter: Marco Colombo
>
> In JdbcRRD it was possible to use a function to get a JDBC connection. This allow an
external handling of the connections while now this is no more possible with dataframes. 
> Please consider an addition to Dataframes for using an externally provided connectionFactory
(such as a connection pool) in order to make data loading more efficient, avoiding connection
close/recreation. Connections should be taken from provided function and returned to a second
function whenever no more used by the RRD. So this will make jdbc handling more efficient.
> I.e. extending DataFrame class with a method like jdbc(Function0<java.sql.Connection>
getConnection, Function0 releaseConnection(java.sql.Connection))



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message