hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: distribute objects prior to launching MR
Date Sat, 23 Apr 2011 08:52:18 GMT
Hello Abhay,

On Sat, Apr 23, 2011 at 11:32 AM, abhay ratnaparkhi
<abhay.ratnaparkhi@gmail.com> wrote:
> Is it possible to get data from database before launching job and then pass
> the object to all Maps?
> I know we can use DistributedCache to distribute files. But is there any
> facility to distribute objects?

There's no direct way of doing that in Hadoop without manual
ser-deser. Serialize into a file -> Add to DC -> Deserialize in Tasks,
should do it.

Harsh J

View raw message