hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arindam Khaled <akha...@utdallas.edu>
Subject Re: Moving Files to Distributed Cache in MapReduce
Date Fri, 29 Jul 2011 18:55:37 GMT
Please unsubscribe me.

On Jul 29, 2011, at 1:18 PM, Mapred Learn wrote:

> I hope my previous reply helps...
>
> On Fri, Jul 29, 2011 at 11:11 AM, Roger Chen <rogchen@ucdavis.edu>  
> wrote:
>
>> After moving it to the distributed cache, how would I call it  
>> within my
>> MapReduce program?
>>
>> On Fri, Jul 29, 2011 at 11:09 AM, Mapred Learn  
>> <mapred.learn@gmail.com
>>> wrote:
>>
>>> Did you try using -files option in your hadoop jar command as:
>>>
>>> /usr/bin/hadoop jar <jar name> <main class name> -files  <absolute
 
>>> path
>> of
>>> file to be added to distributed cache> <input dir> <output dir>
>>>
>>>
>>> On Fri, Jul 29, 2011 at 11:05 AM, Roger Chen <rogchen@ucdavis.edu>
>> wrote:
>>>
>>>> Slight modification: I now know how to add files to the distributed
>> file
>>>> cache, which can be done via this command placed in the main or run
>>> class:
>>>>
>>>>       DistributedCache.addCacheFile(new
>> URI("/user/hadoop/thefile.dat"),
>>>> conf);
>>>>
>>>> However I am still having trouble locating the file in the  
>>>> distributed
>>>> cache. *How do I call the file path of thefile.dat in the  
>>>> distributed
>>> cache
>>>> as a string?* I am using Hadoop 0.20.2
>>>>
>>>>
>>>> On Fri, Jul 29, 2011 at 10:26 AM, Roger Chen <rogchen@ucdavis.edu>
>>> wrote:
>>>>
>>>>> Hi all,
>>>>>
>>>>> Does anybody have examples of how one moves files from the local
>>>>> filestructure/HDFS to the distributed cache in MapReduce? A Google
>>> search
>>>>> turned up examples in Pig but not MR.
>>>>>
>>>>> --
>>>>> Roger Chen
>>>>> UC Davis Genome Center
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Roger Chen
>>>> UC Davis Genome Center
>>>>
>>>
>>
>>
>>
>> --
>> Roger Chen
>> UC Davis Genome Center
>>


Mime
View raw message