spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "clouds (JIRA)" <>
Subject [jira] [Created] (SPARK-26383) NPE when use DataFrameReader.jdbc with wrong URL
Date Mon, 17 Dec 2018 11:01:00 GMT
clouds created SPARK-26383:

             Summary: NPE when use DataFrameReader.jdbc with wrong URL
                 Key: SPARK-26383
             Project: Spark
          Issue Type: Improvement
          Components: Spark Core
    Affects Versions: 2.4.0
            Reporter: clouds

When passing wrong url to jdbc:
val opts = Map(
  "url" -> "jdbc:mysql://localhost/db",
  "dbtable" -> "table",
  "driver" -> "org.postgresql.Driver"
var df ="jdbc").options(opts).load
It would throw an NPE instead of complaining about connection failed. (Note url and driver
not matched here)
Exception in thread "main" java.lang.NullPointerException
    at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:71)
    at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
    at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
    at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
    at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
    at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
as [postgresql jdbc driver document|] saying, The
driver should return "null" if it realizes it is the wrong kind of driver to connect to the
given URL.

while [ConnectionFactory|] would
not check if conn is null.
val conn: Connection = JdbcUtils.createConnectionFactory(options)()
 and trying to close the conn anyway
try {
} finally {

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message