hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geoffry Roberts <geoffry.robe...@gmail.com>
Subject Re: Passing an Object to All Reducers
Date Fri, 06 May 2011 17:58:40 GMT
Steve,

Yes the object is known at start up and is read only.  The mappers don't
touch it.  I considered serializing to a string.  I was wondering if there
wasn't a more excellent way.

Thanks

On 6 May 2011 10:55, Steve Lewis <lordjoe2000@gmail.com> wrote:

> If possible serialize the object as XML then add it as a set of lines to
> the config - alternatively serialize it (maybe xml) to a known spot in HDFS
> and read it in in the setup code in the reducer - I assume this is an object
> known at the start of the job and not modified by the mapper
>
>
> On Fri, May 6, 2011 at 10:12 AM, Geoffry Roberts <
> geoffry.roberts@gmail.com> wrote:
>
>> All,
>>
>> I need for each one of my reducers to have read access to a certain object
>> or a clone thereof.  I can instantiate this object a start up.  How can I
>> give my reducers a copy?
>>
>> --
>> Geoffry Roberts
>>
>>
>
>
> --
> Steven M. Lewis PhD
> 4221 105th Ave NE
> Kirkland, WA 98033
> 206-384-1340 (cell)
> Skype lordjoe_com
>
>
>


-- 
Geoffry Roberts

Mime
View raw message