hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shahab Yunus <shahab.yu...@gmail.com>
Subject Re: New Distributed Cache
Date Thu, 11 Jul 2013 01:43:16 GMT
Also, once you have the array of URIs after calling getCacheFiles  you can
iterate over them using File class or Path (
http://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/Path.html#Path(java.net.URI)
)

Regards,
Shahab


On Wed, Jul 10, 2013 at 5:08 PM, Omkar Joshi <ojoshi@hortonworks.com> wrote:

> did you try JobContext.getCacheFiles() ?
>
>
> Thanks,
> Omkar Joshi
> *Hortonworks Inc.* <http://www.hortonworks.com>
>
>
> On Wed, Jul 10, 2013 at 10:15 AM, Botelho, Andrew <Andrew.Botelho@emc.com>wrote:
>
>> Hi,****
>>
>> ** **
>>
>> I am trying to store a file in the Distributed Cache during my Hadoop job.
>> ****
>>
>> In the driver class, I tell the job to store the file in the cache with
>> this code:****
>>
>> ** **
>>
>> Job job = Job.getInstance();****
>>
>> job.addCacheFile(new URI("file name"));****
>>
>> ** **
>>
>> That all compiles fine.  In the Mapper code, I try accessing the cached
>> file with this method:****
>>
>> ** **
>>
>> Path[] localPaths = context.getLocalCacheFiles();****
>>
>> ** **
>>
>> However, I am getting warnings that this method is deprecated.****
>>
>> Does anyone know the newest way to access cached files in the Mapper
>> code? (I am using Hadoop 2.0.5)****
>>
>> ** **
>>
>> Thanks in advance,****
>>
>> ** **
>>
>> Andrew****
>>
>
>

Mime
View raw message