spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "clouds (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-26383) NPE when use DataFrameReader.jdbc with wrong URL
Date Mon, 17 Dec 2018 11:01:00 GMT
clouds created SPARK-26383:
------------------------------

             Summary: NPE when use DataFrameReader.jdbc with wrong URL
                 Key: SPARK-26383
                 URL: https://issues.apache.org/jira/browse/SPARK-26383
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 2.4.0
            Reporter: clouds


When passing wrong url to jdbc:
{code:java}
val opts = Map(
  "url" -> "jdbc:mysql://localhost/db",
  "dbtable" -> "table",
  "driver" -> "org.postgresql.Driver"
)
var df = spark.read.format("jdbc").options(opts).load
{code}
It would throw an NPE instead of complaining about connection failed. (Note url and driver
not matched here)
{code:java}
Exception in thread "main" java.lang.NullPointerException
    at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:71)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
    at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
    at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
    at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
{code}
as [postgresql jdbc driver document|https://jdbc.postgresql.org/development/privateapi/org/postgresql/Driver.html#connect-java.lang.String-java.util.Properties-] saying, The
driver should return "null" if it realizes it is the wrong kind of driver to connect to the
given URL.

while [ConnectionFactory|https://github.com/apache/spark/blob/e743e848484bf7d97e1b4f33ea83f8520ae7da04/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRDD.scala#L56] would
not check if conn is null.
{code:java}
val conn: Connection = JdbcUtils.createConnectionFactory(options)()
{code}
 and trying to close the conn anyway
{code:java}
try {
  ...
} finally {
  conn.close()
}
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message