kudu-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From granthe...@apache.org
Subject [1/2] kudu git commit: KUDU-2599: fix flaky DefaultSourceTest.testSocketReadTimeoutPropagation
Date Tue, 13 Nov 2018 14:29:10 GMT
Repository: kudu
Updated Branches:
  refs/heads/master 1e27c9fc4 -> c136efe47


KUDU-2599: fix flaky DefaultSourceTest.testSocketReadTimeoutPropagation

This test is just about checking the propagation of an option from the
DefaultSource to the KuduRelation. However, building a DataFrame involves a
connection to the cluster, and if we set socketReadTimeoutMs to 1, it's
going to be very difficult to connect, especially if we're testing
TSAN-instrumented binaries whose thread creation is inherently slower.

The fix? Use a magic number larger than '1'. It's that simple.

Change-Id: I81a20f776c115caf3bd720bc532bbeb5fc69bc42
Reviewed-on: http://gerrit.cloudera.org:8080/11916
Reviewed-by: Andrew Wong <awong@cloudera.com>
Tested-by: Kudu Jenkins
Reviewed-by: Grant Henke <granthenke@apache.org>


Project: http://git-wip-us.apache.org/repos/asf/kudu/repo
Commit: http://git-wip-us.apache.org/repos/asf/kudu/commit/51736f33
Tree: http://git-wip-us.apache.org/repos/asf/kudu/tree/51736f33
Diff: http://git-wip-us.apache.org/repos/asf/kudu/diff/51736f33

Branch: refs/heads/master
Commit: 51736f330935a516c14e923e286c7b3f7e73de02
Parents: 1e27c9f
Author: Adar Dembo <adar@cloudera.com>
Authored: Thu Nov 8 21:41:24 2018 -0800
Committer: Grant Henke <granthenke@apache.org>
Committed: Tue Nov 13 14:27:39 2018 +0000

----------------------------------------------------------------------
 .../org/apache/kudu/spark/kudu/DefaultSourceTest.scala  | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/kudu/blob/51736f33/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala
----------------------------------------------------------------------
diff --git a/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala
b/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala
index bc0d93c..0684a04 100644
--- a/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala
+++ b/java/kudu-spark/src/test/scala/org/apache/kudu/spark/kudu/DefaultSourceTest.scala
@@ -904,10 +904,10 @@ class DefaultSourceTest extends KuduTestSuite with Matchers {
     kuduOptions = Map(
       "kudu.table" -> tableName,
       "kudu.master" -> harness.getMasterAddressesAsString,
-      "kudu.scanRequestTimeoutMs" -> "1")
+      "kudu.scanRequestTimeoutMs" -> "66666")
     val dataFrame = sqlContext.read.options(kuduOptions).kudu
     val kuduRelation = kuduRelationFromDataFrame(dataFrame)
-    assert(kuduRelation.readOptions.scanRequestTimeoutMs == Some(1))
+    assert(kuduRelation.readOptions.scanRequestTimeoutMs == Some(66666))
   }
 
   /**
@@ -920,9 +920,13 @@ class DefaultSourceTest extends KuduTestSuite with Matchers {
     kuduOptions = Map(
       "kudu.table" -> tableName,
       "kudu.master" -> harness.getMasterAddressesAsString,
-      "kudu.socketReadTimeoutMs" -> "1")
+      "kudu.socketReadTimeoutMs" -> "66666")
+    // Even though we're ostensibly just checking for option propagation, the
+    // construction of 'dataFrame' involves a connection to the Kudu cluster,
+    // which is affected by the value of socketReadTimeoutMs. Therefore we must
+    // choose a value that's high enough to actually support establishing a connection.
     val dataFrame = sqlContext.read.options(kuduOptions).kudu
     val kuduRelation = kuduRelationFromDataFrame(dataFrame)
-    assert(kuduRelation.readOptions.socketReadTimeoutMs == Some(1))
+    assert(kuduRelation.readOptions.socketReadTimeoutMs == Some(66666))
   }
 }


Mime
View raw message