hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hello World <snowlo...@gmail.com>
Subject Re: Can I share datas for several map tasks?
Date Tue, 16 Jun 2009 02:53:23 GMT
Thanks for your reply. I tried as you said that init some data in setup()
and read them from map(), it did work, but it still seems that each map task
did its own setup() seperately and they are using different datas, so this
does not match what I need. I want to share some common datas amongst
different map tasks (these datas are read-only after initialization). So is
there any other way?


On Tue, Jun 16, 2009 at 9:28 AM, Eason.Lee <leongfans@gmail.com> wrote:

> I think you can just define the data structures in your map classinit it in
> setup(Context context) and use it in your map method
> hope it is helpful!
>
> 2009/6/16 snowloong <snowloong@gmail.com>
>
> > Hi,
> > I want to share some data structures for the map tasks on a same node(not
> > through files), I mean, if one map task has already initialized some data
> > structures (e.g. an array or a list), can other map tasks share these
> > memorys and directly access them, for I don't want to reinitialize these
> > datas and I want to save some memory. Can hadoop help me do this?
> > thanks
> >
> > 2009-06-16
> >
> >
> >
> > snowloong
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message