hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laurent Hatier <laurent.hat...@gmail.com>
Subject Re: Hadoop problem
Date Mon, 30 May 2011 08:17:46 GMT
Hi everybody,

I have a little problem with cassandra-all jar file : when i want to write
back the result of the MapReduce in DB, he says me that the SpecificRecord
class (Hector API) is not found... I have already check this dependency and
it's ok. Do I have to use the Cassandra API or it's a technical problem ?


2011/5/27 Laurent Hatier <laurent.hatier@gmail.com>

> Of couurse !!! It's logical.
> Thank you John.
> 2011/5/27 John Armstrong <john.armstrong@ccri.com>
>> On Fri, 27 May 2011 13:52:04 +0200, Laurent Hatier
>> <laurent.hatier@gmail.com> wrote:
>> > I'm a newbie with Hadoop/MapReduce. I've a problem with hadoop. I set
>> some
>> > variables in the run function but when Map running, he can't get the
>> value
>> > of theses variables...
>> > If anyone knows the solution :)
>> By the "run function" do you mean the main method that launches the
>> map/reduce job?  It's no surprise that the mappers (and reducers) won't
>> know those variables, because they run as completely separate tasks.
>> If you're computing something in the setup method for use in the mappers
>> or reducers you'll have to pass that information along somehow.  If it's a
>> String (or something that can easily be made into a String, like an int)
>> you can set it as a property in the job's Configuration.  For more
>> complicated data you'll have to serialize it to a file, place the file
>> into
>> the distributed cache, and then deserialize the data within the mapper or
>> reducer's setup method.
>> Of course, if the computation is less complicated/time consuming than the
>> deserialization process, you may as well just recompute the data in each
>> mapper or reducer.
> --
> Laurent HATIER
> Étudiant en 2e année du Cycle Ingénieur à l'EISTI

Laurent HATIER
Étudiant en 2e année du Cycle Ingénieur à l'EISTI

View raw message