hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Heshan Lin <hl...@ncsu.edu>
Subject Re: How to configure RandomWriter to generate less amount of data
Date Tue, 01 Jul 2008 14:25:34 GMT
Thanks a bunch, Amar! It works fine.

/Heshan


On Jul 1, 2008, at 1:44 AM, Amar Kamat wrote:

> Heshan Lin wrote:
>> Hi,
>>
>> I'm trying to configure RandomWriter to generate less data than  
>> does the default configuration.
> bin/hadoop jar hadoop-*-examples.jar randomwriter - 
> Dtest.randomwrite.bytes_per_map=<value> - 
> Dtest.randomwrite.total_bytes=<value> - 
> Dtest.randomwriter.maps_per_host=<value> <input-filename>
> The number of maps that will be spawned in this case will be  
> total_bytes/bytes_per_map.
> Other parameters are test.randomwrite.min_key (size in bytes),  
> test.randomwrite.max_key (size in bytes),  
> test.randomwrite.min_value (size in bytes) and  
> test.randomwrite.max_value (size in bytes).
> Amar
>> I created a job configuration file job.xml and added in variables  
>> given at http://wiki.apache.org/hadoop/RandomWriter. Tried a  
>> couple of ways of running the program below, but configurations in  
>> job.xml were not taken by RandomWriter.
>>
>> 1) bin/hadoop jar hadoop-*-examples.jar randomwriter rand job.xml
>> 2) bin/hadoop jar hadoop-*-examples.jar randomwriter rand --conf  
>> job.xml
>> 3) bin/hadoop jar --conf job.xml hadoop-*-examples.jar  
>> randomwriter rand
>>
>> Passing property values via the -D option didn't seem to work  
>> either. Can anybody advise on how to use the job configuration  
>> file properly?
>>
>> Thanks,
>> Heshan
>


Mime
View raw message