spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject spark git commit: [SPARK-8289] Specify stack size for consistency with Java tests - resolves test failures
Date Thu, 11 Jun 2015 07:41:11 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.4 5c05b5c0d -> b313920ab

[SPARK-8289] Specify stack size for consistency with Java tests - resolves test failures

This change is a simple one and specifies a stack size of 4096k instead of the vendor default
for Java tests (the defaults vary between Java vendors). This remedies test failures observed
with JavaALSSuite with IBM and Oracle Java owing to a lower default size in comparison to
the size with OpenJDK. 4096k is a suitable default where the tests pass with each Java vendor
tested. The alternative is to reduce the number of iterations in the test (no observed failures
with 5 iterations instead of 15).

-Xss works with Oracle's HotSpot VM, IBM's J9 VM and OpenJDK (IcedTea).

I have ensured this does not have any negative implications for other tests.

Author: Adam Roberts <>
Author: a-roberts <>

Closes #6727 from a-roberts/IncJavaStackSize and squashes the following commits:

ab40aea [Adam Roberts] Specify stack size for SBT builds
5032d8d [a-roberts] Update pom.xml

(cherry picked from commit 6b68366df345d4572cf138f9efe17e23d0d1971e)
Signed-off-by: Sean Owen <>


Branch: refs/heads/branch-1.4
Commit: b313920abdc2c075ab038bf38beb62425df0908e
Parents: 5c05b5c
Author: Adam Roberts <>
Authored: Thu Jun 11 08:40:46 2015 +0100
Committer: Sean Owen <>
Committed: Thu Jun 11 08:41:00 2015 +0100

 pom.xml                  | 2 +-
 project/SparkBuild.scala | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/pom.xml b/pom.xml
index f58b9df..e7e2ac9 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1242,7 +1242,7 @@
-            <argLine>-Xmx3g -XX:MaxPermSize=${MaxPermGen} -XX:ReservedCodeCacheSize=512m</argLine>
+            <argLine>-Xmx3g -Xss4096k -XX:MaxPermSize=${MaxPermGen} -XX:ReservedCodeCacheSize=512m</argLine>
                 Setting SPARK_DIST_CLASSPATH is a simple way to make sure any child processes
diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala
index e4ccb5c..0b82a02 100644
--- a/project/SparkBuild.scala
+++ b/project/SparkBuild.scala
@@ -524,7 +524,7 @@ object TestSettings {
     javaOptions in Test ++= System.getProperties.filter(_._1 startsWith "spark")
       .map { case (k,v) => s"-D$k=$v" }.toSeq,
     javaOptions in Test += "-ea",
-    javaOptions in Test ++= "-Xmx3g -XX:PermSize=128M -XX:MaxNewSize=256m -XX:MaxPermSize=1g"
+    javaOptions in Test ++= "-Xmx3g -Xss4096k -XX:PermSize=128M -XX:MaxNewSize=256m -XX:MaxPermSize=1g"
       .split(" ").toSeq,
     javaOptions += "-Xmx3g",
     // Show full stack trace and duration in test cases.

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message