spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <van...@cloudera.com>
Subject SparkSubmit and --driver-java-options
Date Wed, 30 Apr 2014 18:29:03 GMT
Hello all,

Maybe my brain is not evolved enough to be able to trace through what
happens with command-line arguments as they're parsed through all the
shell scripts... but I really can't figure out how to pass more than a
single JVM option on the command line.

Unless someone has an obvious workaround that I'm missing, I'd like to
propose something that is actually pretty standard in JVM tools: using
-J. From javac:

  -J<flag>                   Pass <flag> directly to the runtime system

So "javac -J-Xmx1g" would pass "-Xmx1g" to the underlying JVM. You can
use several of those to pass multiple options (unlike
--driver-java-options), so it helps that it's a short syntax.

Unless someone has some issue with that I'll work on a patch for it...
(well, I'm going to do it locally for me anyway because I really can't
figure out how to do what I want to otherwise.)


-- 
Marcelo

Mime
View raw message