Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 76610200D2B for ; Thu, 19 Oct 2017 06:07:23 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 74EF3160BEB; Thu, 19 Oct 2017 04:07:23 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id BA6CD160BEA for ; Thu, 19 Oct 2017 06:07:22 +0200 (CEST) Received: (qmail 93723 invoked by uid 500); 19 Oct 2017 04:07:21 -0000 Mailing-List: contact commits-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list commits@spark.apache.org Received: (qmail 93714 invoked by uid 99); 19 Oct 2017 04:07:21 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 Oct 2017 04:07:21 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 9DDADDFB0E; Thu, 19 Oct 2017 04:07:21 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: gurwls223@apache.org To: commits@spark.apache.org Message-Id: <448193a1f45c427a91d9bc19d00d4da4@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: spark git commit: [SPARK-21551][PYTHON] Increase timeout for PythonRDD.serveIterator Date: Thu, 19 Oct 2017 04:07:21 +0000 (UTC) archived-at: Thu, 19 Oct 2017 04:07:23 -0000 Repository: spark Updated Branches: refs/heads/branch-2.2 010b50cea -> f8c83fdc5 [SPARK-21551][PYTHON] Increase timeout for PythonRDD.serveIterator Backport of https://github.com/apache/spark/pull/18752 (https://issues.apache.org/jira/browse/SPARK-21551) (cherry picked from commit 9d3c6640f56e3e4fd195d3ad8cead09df67a72c7) Author: peay Closes #19512 from FRosner/branch-2.2. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/f8c83fdc Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/f8c83fdc Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/f8c83fdc Branch: refs/heads/branch-2.2 Commit: f8c83fdc52ba9120098e52a35085448150af6b50 Parents: 010b50c Author: peay Authored: Thu Oct 19 13:07:04 2017 +0900 Committer: hyukjinkwon Committed: Thu Oct 19 13:07:04 2017 +0900 ---------------------------------------------------------------------- .../src/main/scala/org/apache/spark/api/python/PythonRDD.scala | 6 +++--- python/pyspark/rdd.py | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/f8c83fdc/core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala ---------------------------------------------------------------------- diff --git a/core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala b/core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala index b0dd2fc..807b51f 100644 --- a/core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala +++ b/core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala @@ -683,7 +683,7 @@ private[spark] object PythonRDD extends Logging { * Create a socket server and a background thread to serve the data in `items`, * * The socket server can only accept one connection, or close if no connection - * in 3 seconds. + * in 15 seconds. * * Once a connection comes in, it tries to serialize all the data in `items` * and send them into this connection. @@ -692,8 +692,8 @@ private[spark] object PythonRDD extends Logging { */ def serveIterator[T](items: Iterator[T], threadName: String): Int = { val serverSocket = new ServerSocket(0, 1, InetAddress.getByName("localhost")) - // Close the socket if no connection in 3 seconds - serverSocket.setSoTimeout(3000) + // Close the socket if no connection in 15 seconds + serverSocket.setSoTimeout(15000) new Thread(threadName) { setDaemon(true) http://git-wip-us.apache.org/repos/asf/spark/blob/f8c83fdc/python/pyspark/rdd.py ---------------------------------------------------------------------- diff --git a/python/pyspark/rdd.py b/python/pyspark/rdd.py index 6014179..aca00bc 100644 --- a/python/pyspark/rdd.py +++ b/python/pyspark/rdd.py @@ -127,7 +127,7 @@ def _load_from_socket(port, serializer): af, socktype, proto, canonname, sa = res sock = socket.socket(af, socktype, proto) try: - sock.settimeout(3) + sock.settimeout(15) sock.connect(sa) except socket.error: sock.close() --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org For additional commands, e-mail: commits-help@spark.apache.org