hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laurent Hatier <laurent.hat...@gmail.com>
Subject Re: Hadoop problem
Date Fri, 27 May 2011 15:40:56 GMT
Of couurse !!! It's logical.
Thank you John.

2011/5/27 John Armstrong <john.armstrong@ccri.com>

> On Fri, 27 May 2011 13:52:04 +0200, Laurent Hatier
> <laurent.hatier@gmail.com> wrote:
> > I'm a newbie with Hadoop/MapReduce. I've a problem with hadoop. I set
> some
> > variables in the run function but when Map running, he can't get the
> value
> > of theses variables...
> > If anyone knows the solution :)
>
> By the "run function" do you mean the main method that launches the
> map/reduce job?  It's no surprise that the mappers (and reducers) won't
> know those variables, because they run as completely separate tasks.
>
> If you're computing something in the setup method for use in the mappers
> or reducers you'll have to pass that information along somehow.  If it's a
> String (or something that can easily be made into a String, like an int)
> you can set it as a property in the job's Configuration.  For more
> complicated data you'll have to serialize it to a file, place the file into
> the distributed cache, and then deserialize the data within the mapper or
> reducer's setup method.
>
> Of course, if the computation is less complicated/time consuming than the
> deserialization process, you may as well just recompute the data in each
> mapper or reducer.
>



-- 
Laurent HATIER
Étudiant en 2e année du Cycle Ingénieur à l'EISTI

Mime
View raw message