pig-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From z..@apache.org
Subject svn commit: r1786612 - in /pig/branches/spark/test/e2e/pig: conf/spark.conf drivers/TestDriverPig.pm
Date Sun, 12 Mar 2017 21:08:54 GMT
Author: zly
Date: Sun Mar 12 21:08:54 2017
New Revision: 1786612

URL: http://svn.apache.org/viewvc?rev=1786612&view=rev
Log:
PIG-5187:Fix configuration for E2E spark mode (Adam via Liyun)

Modified:
    pig/branches/spark/test/e2e/pig/conf/spark.conf
    pig/branches/spark/test/e2e/pig/drivers/TestDriverPig.pm

Modified: pig/branches/spark/test/e2e/pig/conf/spark.conf
URL: http://svn.apache.org/viewvc/pig/branches/spark/test/e2e/pig/conf/spark.conf?rev=1786612&r1=1786611&r2=1786612&view=diff
==============================================================================
--- pig/branches/spark/test/e2e/pig/conf/spark.conf (original)
+++ pig/branches/spark/test/e2e/pig/conf/spark.conf Sun Mar 12 21:08:54 2017
@@ -30,7 +30,7 @@ my $hdfsBase = $ENV{PH_HDFS_BASE} || "/u
 
 $cfg = {
     #HDFS
-      'inpathbase'     => "$hdfsBase/test/data"
+      'inpathbase'     => "$hdfsBase/tests/data"
     , 'outpathbase'    => "$hdfsBase/out"
 
    #LOCAL

Modified: pig/branches/spark/test/e2e/pig/drivers/TestDriverPig.pm
URL: http://svn.apache.org/viewvc/pig/branches/spark/test/e2e/pig/drivers/TestDriverPig.pm?rev=1786612&r1=1786611&r2=1786612&view=diff
==============================================================================
--- pig/branches/spark/test/e2e/pig/drivers/TestDriverPig.pm (original)
+++ pig/branches/spark/test/e2e/pig/drivers/TestDriverPig.pm Sun Mar 12 21:08:54 2017
@@ -425,6 +425,11 @@ sub getPigCmd($$$)
         }
         TestDriver::dbg("Additional java parameters: [$additionalJavaParams].\n");
     }
+
+    # Several OutOfMemoryErrors - Perm space issues were seen during running E2E tests, here
max Perm size is adjusted
+    if ($testCmd->{'exectype'} eq "spark") {
+        $additionalJavaParams = "-XX:MaxPermSize=512m";
+    }
     
     push(@pigCmd, ("-x", $testCmd->{'exectype'}));
 



Mime
View raw message