spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject spark git commit: [SPARK-14204][SQL] register driverClass rather than user-specified class
Date Wed, 03 Aug 2016 20:15:16 GMT
Repository: spark
Updated Branches:
  refs/heads/master e6f226c56 -> 685b08e26

[SPARK-14204][SQL] register driverClass rather than user-specified class

This is a pull request that was originally merged against branch-1.6 as #12000, now being
merged into master as well.  srowen zzcclp JoshRosen

This pull request fixes an issue in which cluster-mode executors fail to properly register
a JDBC driver when the driver is provided in a jar by the user, but the driver class name
is derived from a JDBC URL (rather than specified by the user). The consequence of this is
that all JDBC accesses under the described circumstances fail with an IllegalStateException.
I reported the issue here:

My proposed solution is to have the executors register the JDBC driver class under all circumstances,
not only when the driver is specified by the user.

This patch was tested manually. I built an assembly jar, deployed it to a cluster, and confirmed
that the problem was fixed.

Author: Kevin McHale <>

Closes #14420 from mchalek/mchalek-jdbc_driver_registration.


Branch: refs/heads/master
Commit: 685b08e2611b69f8db60a00c0c94aecd315e2a3e
Parents: e6f226c
Author: Kevin McHale <>
Authored: Wed Aug 3 13:15:13 2016 -0700
Committer: Sean Owen <>
Committed: Wed Aug 3 13:15:13 2016 -0700

 .../apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala    | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala
index 81d38e3..a33c26d 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala
@@ -55,7 +55,7 @@ object JdbcUtils extends Logging {
     () => {
-      userSpecifiedDriverClass.foreach(DriverRegistry.register)
+      DriverRegistry.register(driverClass)
       val driver: Driver = DriverManager.getDrivers.asScala.collectFirst {
         case d: DriverWrapper if d.wrapped.getClass.getCanonicalName == driverClass =>
         case d if d.getClass.getCanonicalName == driverClass => d

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message