hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Giordano, Michael" <Michael.Giord...@Vistronix.com>
Subject RE: DistributedCache deprecated
Date Wed, 29 Jan 2014 22:53:22 GMT

Thank you for the prompt answer. I see the methods on the job class and this does make sense.

Unfortunately something else has me confused. It seems as though all of the Job() constructors
have also been marked deprecated.


How do you create a new Job instance? Is there a factory class?


Mike G.

From: praveenesh kumar <praveenesh@gmail.com>
Sent: Wednesday, January 29, 2014 4:41 PM
To: user@hadoop.apache.org
Subject: Re: DistributedCache deprecated

@Jay - I don't know how Job class is replacing the DistributedCache class , but I remember
trying distributed cache functions like

 void   addArchiveToClassPath<http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/mapreduce/Job.html#addArchiveToClassPath%28org.apache.hadoop.fs.Path%29>(Path<http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/fs/Path.html>
          Add an archive path to the current set of classpath entries.
 void   addCacheArchive<http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/mapreduce/Job.html#addCacheArchive%28java.net.URI%29>(URI<http://download.oracle.com/javase/6/docs/api/java/net/URI.html?is-external=true>
          Add a archives to be localized
 void   addCacheFile<http://hadoop.apache.org/docs/stable2/api/org/apache/hadoop/mapreduce/Job.html#addCacheFile%28java.net.URI%29>(URI<http://download.oracle.com/javase/6/docs/api/java/net/URI.html?is-external=true>
          Add a file to be localized

and it works fine. The same way you were using DC before.. Well I am not sure what would be
the best answer, but if you are trying to use DC , I was able to do it with Job class itself.


On Wed, Jan 29, 2014 at 9:27 PM, Jay Vyas <jayunit100@gmail.com<mailto:jayunit100@gmail.com>>
Thanks for asking this : Im not sure and didnt realize this until you mentioned it!

1) Prav:  You are implying that we would use the "Job" Class... but how could it replace the

2) The point of the DC is to replicate a file so that its present and local on ALL nodes.
  I didnt know it was deprecated, but there must be some replacement for it - or maybe it
just got renamed and moved?

SO ... what is the future of the DistributedCache for mapreduce jobs?

On Wed, Jan 29, 2014 at 4:22 PM, praveenesh kumar <praveenesh@gmail.com<mailto:praveenesh@gmail.com>>
I think you can use the Job class.


On Wed, Jan 29, 2014 at 9:13 PM, Giordano, Michael <Michael.Giordano@vistronix.com<mailto:Michael.Giordano@vistronix.com>>

I noticed that in Hadoop 2.2.0 org.apache.hadoop.mapreduce.filecache.DistributedCache has
been deprecated.


Is there a class that provides equivalent functionality? My application relies heavily on


Mike G.

This communication, along with its attachments, is considered confidential and proprietary
to Vistronix.  It is intended only for the use of the person(s) named above.  Note that unauthorized
disclosure or distribution of information not generally known to the public is strictly prohibited.
 If you are not the intended recipient, please notify the sender immediately.

Jay Vyas

View raw message