hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sudharsan Sampath <sudha...@gmail.com>
Subject Re: Can I define a datastructure that all Mappers share?
Date Mon, 15 Aug 2011 10:25:06 GMT

To my knowledge, its not possible with plain map-reduce. But you can try
using a distributed cache on top of it. To quote a few try, hazelcast (if ur
prog lang is java) or gigaspace.

Just a note, why would you want to share date across mappers. It defeats the
basic assumption of map-reduce that mappers are independent work chunks.

-Sudhan S

2011/8/15 谭军 <tanjun_2525@163.com>

> Hi,
> I want to define a matrix or list that all mappers share.
> So that all mappers can do operations on it.
> Can I make it?
> Thanks!
> --
> Regards!
> Jun Tan

View raw message