hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From praveenesh kumar <praveen...@gmail.com>
Subject Re: DistributedCache deprecated
Date Wed, 29 Jan 2014 21:41:09 GMT
@Jay - I don't know how Job class is replacing the DistributedCache class ,
but I remember trying distributed cache functions like

   void *addArchiveToClassPath
<http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/mapreduce/Job.html#addArchiveToClassPath%28org.apache.hadoop.fs.Path%29>*
(Path<http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/fs/Path.html>
 archive)
          Add an archive path to the current set of classpath entries.
 void *addCacheArchive
<http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/mapreduce/Job.html#addCacheArchive%28java.net.URI%29>*
(URI<http://download.oracle.com/javase/6/docs/api/java/net/URI.html?is-external=true>
 uri)
          Add a archives to be localized   void *addCacheFile
<http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/mapreduce/Job.html#addCacheFile%28java.net.URI%29>*
(URI<http://download.oracle.com/javase/6/docs/api/java/net/URI.html?is-external=true>
 uri)
          Add a file to be localized

and it works fine. The same way you were using DC before.. Well I am not
sure what would be the best answer, but if you are trying to use DC , I was
able to do it with Job class itself.

Regards
Prav


On Wed, Jan 29, 2014 at 9:27 PM, Jay Vyas <jayunit100@gmail.com> wrote:

> Thanks for asking this : Im not sure and didnt realize this until you
> mentioned it!
>
> 1) Prav:  You are implying that we would use the "Job" Class... but how
> could it replace the DC?
>
> 2) The point of the DC is to replicate a file so that its present and
> local on ALL nodes.   I didnt know it was deprecated, but there must be
> some replacement for it - or maybe it just got renamed and moved?
>
> SO ... what is the future of the DistributedCache for mapreduce jobs?
>
>
> On Wed, Jan 29, 2014 at 4:22 PM, praveenesh kumar <praveenesh@gmail.com>wrote:
>
>> I think you can use the Job class.
>>
>> http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/mapreduce/Job.html
>>
>> Regards
>> Prav
>>
>>
>> On Wed, Jan 29, 2014 at 9:13 PM, Giordano, Michael <
>> Michael.Giordano@vistronix.com> wrote:
>>
>>>  I noticed that in Hadoop 2.2.0
>>> org.apache.hadoop.mapreduce.filecache.DistributedCache has been deprecated.
>>>
>>>
>>>
>>> (http://hadoop.apache.org/docs/current/api/deprecated-list.html#class)
>>>
>>>
>>>
>>> Is there a class that provides equivalent functionality? My application
>>> relies heavily on DistributedCache.
>>>
>>>
>>>
>>> Thanks,
>>>
>>> Mike G.
>>>
>>> This communication, along with its attachments, is considered
>>> confidential and proprietary to Vistronix.  It is intended only for the use
>>> of the person(s) named above.  Note that unauthorized disclosure or
>>> distribution of information not generally known to the public is strictly
>>> prohibited.  If you are not the intended recipient, please notify the
>>> sender immediately.
>>>
>>
>>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
>

Mime
View raw message