spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zsxw...@apache.org
Subject spark git commit: [SPARK-15059][CORE] Remove fine-grained lock in ChildFirstURLClassLoader to avoid dead lock
Date Tue, 03 May 2016 19:22:57 GMT
Repository: spark
Updated Branches:
  refs/heads/master 84b3a4a87 -> f5623b460


[SPARK-15059][CORE] Remove fine-grained lock in ChildFirstURLClassLoader to avoid dead lock

## What changes were proposed in this pull request?

In some cases, fine-grained lock have race condition with class-loader lock and have caused
dead lock issue. It is safe to drop this fine grained lock and load all classes by single
class-loader lock.

Author: Zheng Tan <zheng.tan@hulu.com>

Closes #12857 from tankkyo/master.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/f5623b46
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/f5623b46
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/f5623b46

Branch: refs/heads/master
Commit: f5623b460224ce363316c63f5d28947215078fc5
Parents: 84b3a4a
Author: Zheng Tan <zheng.tan@hulu.com>
Authored: Tue May 3 12:22:52 2016 -0700
Committer: Shixiong Zhu <shixiong@databricks.com>
Committed: Tue May 3 12:22:52 2016 -0700

----------------------------------------------------------------------
 .../spark/util/MutableURLClassLoader.scala      | 31 ++++----------------
 1 file changed, 5 insertions(+), 26 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/f5623b46/core/src/main/scala/org/apache/spark/util/MutableURLClassLoader.scala
----------------------------------------------------------------------
diff --git a/core/src/main/scala/org/apache/spark/util/MutableURLClassLoader.scala b/core/src/main/scala/org/apache/spark/util/MutableURLClassLoader.scala
index 0a3180d..034826c 100644
--- a/core/src/main/scala/org/apache/spark/util/MutableURLClassLoader.scala
+++ b/core/src/main/scala/org/apache/spark/util/MutableURLClassLoader.scala
@@ -19,7 +19,6 @@ package org.apache.spark.util
 
 import java.net.{URL, URLClassLoader}
 import java.util.Enumeration
-import java.util.concurrent.ConcurrentHashMap
 
 import scala.collection.JavaConverters._
 
@@ -48,32 +47,12 @@ private[spark] class ChildFirstURLClassLoader(urls: Array[URL], parent:
ClassLoa
 
   private val parentClassLoader = new ParentClassLoader(parent)
 
-  /**
-   * Used to implement fine-grained class loading locks similar to what is done by Java 7.
This
-   * prevents deadlock issues when using non-hierarchical class loaders.
-   *
-   * Note that due to some issues with implementing class loaders in
-   * Scala, Java 7's `ClassLoader.registerAsParallelCapable` method is not called.
-   */
-  private val locks = new ConcurrentHashMap[String, Object]()
-
   override def loadClass(name: String, resolve: Boolean): Class[_] = {
-    var lock = locks.get(name)
-    if (lock == null) {
-      val newLock = new Object()
-      lock = locks.putIfAbsent(name, newLock)
-      if (lock == null) {
-        lock = newLock
-      }
-    }
-
-    lock.synchronized {
-      try {
-        super.loadClass(name, resolve)
-      } catch {
-        case e: ClassNotFoundException =>
-          parentClassLoader.loadClass(name, resolve)
-      }
+    try {
+      super.loadClass(name, resolve)
+    } catch {
+      case e: ClassNotFoundException =>
+        parentClassLoader.loadClass(name, resolve)
     }
   }
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message