hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Mittal <amitmitt...@gmail.com>
Subject Re: DistributedCache deprecated
Date Thu, 30 Jan 2014 06:12:48 GMT
Hi Mike & Prav,

Although I am new to Hadoop, but would like to add my 2 cents if that helps.
We are having 2 ways for distribution of shared data, one is using Job
configuration and other is DistributedCache.
As job configuration is read by the JT, TT and child JVMs, and each time
the configuration is read, all of its entries are read in memory, even if
they are not used. So using job configuration is not advised if the data is
more than few kilobytes. So it is not alternative to DistributedCache
unless some modifications are done in Job configuration to address this
So I am also curious to know the alternatative to DistributedCache class.


On Thu, Jan 30, 2014 at 2:43 AM, Giordano, Michael <
Michael.Giordano@vistronix.com> wrote:

>  I noticed that in Hadoop 2.2.0
> org.apache.hadoop.mapreduce.filecache.DistributedCache has been deprecated.
> (http://hadoop.apache.org/docs/current/api/deprecated-list.html#class)
> Is there a class that provides equivalent functionality? My application
> relies heavily on DistributedCache.
> Thanks,
> Mike G.
> This communication, along with its attachments, is considered confidential
> and proprietary to Vistronix.  It is intended only for the use of the
> person(s) named above.  Note that unauthorized disclosure or distribution
> of information not generally known to the public is strictly
> prohibited.  If you are not the intended recipient, please notify the
> sender immediately.

View raw message