hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chase Bradford <chase.bradf...@gmail.com>
Subject Re: Best way to access same data into mapper/reducer class
Date Mon, 21 Feb 2011 19:00:59 GMT
I did a write up on this sort of thing, but it was based on HashSets.
You might find it useful:

http://chasebradford.wordpress.com/2011/02/05/distributed-cache-static-objects-and-fast-setup/

On Mon, Feb 21, 2011 at 10:57 AM, Shuja Rehman <shujamughal@gmail.com> wrote:
> Hi All,
>
> I have a simple question. I have a arraylist which i am populating through
> db in main class. now i want to use the same list in my map and reduce class
> so the question is how to access/send this array list in map and reduce
> class. one way is to again fetch the data in mapper and reducer classes
> which means 3 times db access and seems not suitable and other way either
> could be define arraylist as static in main class and then access it or
> might be using distributed cache.
>
> so kindly let me know the best way of doing it
>
> Thanks
> --
> Regards
> Shuja-ur-Rehman Baig
>
>
>



-- 
Chase Bradford


“If in physics there's something you don't understand, you can always
hide behind the uncharted depths of nature. But if your program
doesn't work, there is no obstinate nature. If it doesn't work, you've
messed up.”

- Edsger Dijkstra

Mime
View raw message