hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amar Kamat <ama...@yahoo-inc.com>
Subject Re: How to configure RandomWriter to generate less amount of data
Date Tue, 01 Jul 2008 05:44:23 GMT
Heshan Lin wrote:
> Hi,
>
> I'm trying to configure RandomWriter to generate less data than does 
> the default configuration. 
bin/hadoop jar hadoop-*-examples.jar randomwriter 
-Dtest.randomwrite.bytes_per_map=<value> 
-Dtest.randomwrite.total_bytes=<value> 
-Dtest.randomwriter.maps_per_host=<value> <input-filename>
The number of maps that will be spawned in this case will be 
total_bytes/bytes_per_map.
Other parameters are test.randomwrite.min_key (size in bytes), 
test.randomwrite.max_key (size in bytes), test.randomwrite.min_value 
(size in bytes) and test.randomwrite.max_value (size in bytes).
Amar
> I created a job configuration file job.xml and added in variables 
> given at http://wiki.apache.org/hadoop/RandomWriter. Tried a couple of 
> ways of running the program below, but configurations in job.xml were 
> not taken by RandomWriter.
>
> 1) bin/hadoop jar hadoop-*-examples.jar randomwriter rand job.xml
> 2) bin/hadoop jar hadoop-*-examples.jar randomwriter rand --conf job.xml
> 3) bin/hadoop jar --conf job.xml hadoop-*-examples.jar randomwriter rand
>
> Passing property values via the -D option didn't seem to work either. 
> Can anybody advise on how to use the job configuration file properly?
>
> Thanks,
> Heshan


Mime
View raw message