hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Passing Properties With Whitespace To Streaming
Date Wed, 28 Oct 2009 19:21:03 GMT
Hi Brian,

Any chance you are using the Cloudera distribution? We did accidentally ship
a bug like this which will be ameliorated in our next release.

The temporary workarounds are:

a) edit /usr/bin/hadoop and change the $* to a "$@"  (including the quotes!)
b) use /usr/lib/hadoop-0.20/bin/hadoop instead of /usr/bin/hadoop

The fix (a) will be included in our release due out the end of this week.


On Wed, Oct 28, 2009 at 12:10 PM, Brian Vargas <brian@ardvaark.net> wrote:

> Hi,
> Using Hadoop 0.20 (CDH2) I'm trying to pass some JVM options to my child
> tasks on the command-line, like this:
> $ hadoop jar streaming.jar -D mapred.reduce.tasks=0 -D
> 'mapred.child.java.opts=-Xms200m -Xmx400m' -input foo.txt -output bar
> -mapper /bin/cat
> However, this fails with:
> ERROR streaming.StreamJob: Unrecognized option: -Xmx400m
> This used to work in 0.18 (although using -jobconf instead of -D).  Did
> the generic command-line parser change between 0.18 and 0.20 to further
> split the arguments on whitespace?  If so, how would one pass property
> values containing whitespace?
> Thanks for any thoughts!  Best!
> Brian

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message