spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "joy-m (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-24205) java.util.concurrent.locks.LockSupport.parkNanos
Date Tue, 08 May 2018 02:57:01 GMT
joy-m created SPARK-24205:
-----------------------------

             Summary: java.util.concurrent.locks.LockSupport.parkNanos
                 Key: SPARK-24205
                 URL: https://issues.apache.org/jira/browse/SPARK-24205
             Project: Spark
          Issue Type: Bug
          Components: YARN
    Affects Versions: 2.2.0
            Reporter: joy-m


when i use yarn client mode, the spark task locked in the collect stage
Because of the data is in the driver machine, so I used the client mode to run my application!
but the the stage collect was locked!
countDf.collect().map(_.getLong(0)).mkString.toLong

{{sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
java.util.concurrent.ThreadPoolExecutor.awaitTermination(ThreadPoolExecutor.java:1475) org.apache.spark.rpc.netty.Dispatcher.awaitTermination(Dispatcher.scala:180)
org.apache.spark.rpc.netty.NettyRpcEnv.awaitTermination(NettyRpcEnv.scala:281) org.apache.spark.executor.CoarseGrainedExecutorBackend$$anonfun$run$1.apply$mcV$sp(CoarseGrainedExecutorBackend.scala:231)
org.apache.spark.deploy.SparkHadoopUtil$$anon$2.run(SparkHadoopUtil.scala:67) org.apache.spark.deploy.SparkHadoopUtil$$anon$2.run(SparkHadoopUtil.scala:66)
java.security.AccessController.doPrivileged(Native Method) javax.security.auth.Subject.doAs(Subject.java:422)
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692) org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:66)
org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:188)
org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:284)
org.apache.spark.executor.CoarseGrainedExecutorBackend.main(CoarseGrainedExecutorBackend.scala)}}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message