spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ma...@apache.org
Subject [2/2] git commit: Merge pull request #210 from haitaoyao/http-timeout
Date Thu, 28 Nov 2013 02:24:50 GMT
Merge pull request #210 from haitaoyao/http-timeout

add http timeout for httpbroadcast

While pulling task bytecode from HttpBroadcast server, there's no timeout value set. This
may cause spark executor code hang and other task in the same executor process wait for the
lock. I have encountered the issue in my cluster. Here's the stacktrace I captured  : https://gist.github.com/haitaoyao/7655830

So add a time out value to ensure the task fail fast.


Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/743a31a7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/743a31a7
Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/743a31a7

Branch: refs/heads/master
Commit: 743a31a7ca4421cbbd5b615b773997a06a7ab4ee
Parents: fb6875d db998a6
Author: Matei Zaharia <matei@eecs.berkeley.edu>
Authored: Wed Nov 27 18:24:39 2013 -0800
Committer: Matei Zaharia <matei@eecs.berkeley.edu>
Committed: Wed Nov 27 18:24:39 2013 -0800

----------------------------------------------------------------------
 .../scala/org/apache/spark/broadcast/HttpBroadcast.scala  | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
----------------------------------------------------------------------



Mime
View raw message