spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From t...@apache.org
Subject spark git commit: [SPARK-16722][TESTS] Fix a StreamingContext leak in StreamingContextSuite when eventually fails
Date Tue, 26 Jul 2016 01:26:33 GMT
Repository: spark
Updated Branches:
  refs/heads/master db36e1e75 -> e164a04b2


[SPARK-16722][TESTS] Fix a StreamingContext leak in StreamingContextSuite when eventually
fails

## What changes were proposed in this pull request?

This PR moves `ssc.stop()` into `finally` for `StreamingContextSuite.createValidCheckpoint`
to avoid leaking a StreamingContext since leaking a StreamingContext will fail a lot of tests
and make us hard to find the real failure one.

## How was this patch tested?

Jenkins unit tests

Author: Shixiong Zhu <shixiong@databricks.com>

Closes #14354 from zsxwing/ssc-leak.


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/e164a04b
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/e164a04b
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/e164a04b

Branch: refs/heads/master
Commit: e164a04b2ba3503e5c14cd1cd4beb40e0b79925a
Parents: db36e1e
Author: Shixiong Zhu <shixiong@databricks.com>
Authored: Mon Jul 25 18:26:29 2016 -0700
Committer: Tathagata Das <tathagata.das1565@gmail.com>
Committed: Mon Jul 25 18:26:29 2016 -0700

----------------------------------------------------------------------
 .../org/apache/spark/streaming/StreamingContextSuite.scala  | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/e164a04b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
----------------------------------------------------------------------
diff --git a/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
index 806e181..f1482e5 100644
--- a/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
+++ b/streaming/src/test/scala/org/apache/spark/streaming/StreamingContextSuite.scala
@@ -819,10 +819,13 @@ class StreamingContextSuite extends SparkFunSuite with BeforeAndAfter
with Timeo
     ssc.checkpoint(checkpointDirectory)
     ssc.textFileStream(testDirectory).foreachRDD { rdd => rdd.count() }
     ssc.start()
-    eventually(timeout(10000 millis)) {
-      assert(Checkpoint.getCheckpointFiles(checkpointDirectory).size > 1)
+    try {
+      eventually(timeout(30000 millis)) {
+        assert(Checkpoint.getCheckpointFiles(checkpointDirectory).size > 1)
+      }
+    } finally {
+      ssc.stop()
     }
-    ssc.stop()
     checkpointDirectory
   }
 


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message