hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amareshwari Sriramadasu <amar...@yahoo-inc.com>
Subject Re: streaming question
Date Mon, 15 Sep 2008 04:26:30 GMT
Dennis Kubes wrote:
> If I understand what you are asking you can use the -cacheArchive with 
> the path to the jar to including the jar file in the classpath of your 
> streaming job.
>
> Dennis
>
You can also use -cacheArchive option to include jar file and symlink 
the unjarred directory from cwd by providing the uri as 
hdfs://<path>#link. You have to provide -reducer and -combiner options 
as appropriate paths in the unjarred directory.

Thanks
Amareshwari
> Christian S√łttrup wrote:
>> Hi all,
>>
>> I have an application that i use to run with the "hadoop jar" command.
>> I have now written an optimized version of the mapper in C.
>> I have run this using the streaming library and everything looks ok 
>> (using num.reducers=0).
>>
>> Now i want to use this mapper together with the combiner and reducer 
>> from my old .jar file.
>> How do i do this? How can i distribute the jar and run the reducer 
>> and combiner from it?
>> While also running the c program as the mapper in streaming mode.
>>
>> cheers,
>> Christian
>>
>>


Mime
View raw message