hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eason.Lee" <leongf...@gmail.com>
Subject Re: Can I share datas for several map tasks?
Date Tue, 16 Jun 2009 03:28:42 GMT
Sorry, I think I have mistook your problem.I thought you want share data in
the different calls of the map() method in the same MapTask
Hope anyone else can help you [?]

2009/6/16 Hello World <snowloong@gmail.com>

> Thanks for your reply. I tried as you said that init some data in setup()
> and read them from map(), it did work, but it still seems that each map
> task
> did its own setup() seperately and they are using different datas, so this
> does not match what I need. I want to share some common datas amongst
> different map tasks (these datas are read-only after initialization). So is
> there any other way?
>
>
> On Tue, Jun 16, 2009 at 9:28 AM, Eason.Lee <leongfans@gmail.com> wrote:
>
> > I think you can just define the data structures in your map classinit it
> in
> > setup(Context context) and use it in your map method
> > hope it is helpful!
> >
> > 2009/6/16 snowloong <snowloong@gmail.com>
> >
> > > Hi,
> > > I want to share some data structures for the map tasks on a same
> node(not
> > > through files), I mean, if one map task has already initialized some
> data
> > > structures (e.g. an array or a list), can other map tasks share these
> > > memorys and directly access them, for I don't want to reinitialize
> these
> > > datas and I want to save some memory. Can hadoop help me do this?
> > > thanks
> > >
> > > 2009-06-16
> > >
> > >
> > >
> > > snowloong
> > >
> >
>

Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message