hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Laurent Hatier <laurent.hat...@gmail.com>
Subject Re: Hadoop problem
Date Mon, 30 May 2011 08:29:44 GMT
I have written a mistake : of course SpecificRecord is in the cassandra API
but my question is : why i have this problem ? or Is the an another way to
write back the result ?

Thanks

2011/5/30 Laurent Hatier <laurent.hatier@gmail.com>

> Hi everybody,
>
> I have a little problem with cassandra-all jar file : when i want to write
> back the result of the MapReduce in DB, he says me that the SpecificRecord
> class (Hector API) is not found... I have already check this dependency and
> it's ok. Do I have to use the Cassandra API or it's a technical problem ?
>
> Thanks
>
>
> 2011/5/27 Laurent Hatier <laurent.hatier@gmail.com>
>
>> Of couurse !!! It's logical.
>> Thank you John.
>>
>> 2011/5/27 John Armstrong <john.armstrong@ccri.com>
>>
>>> On Fri, 27 May 2011 13:52:04 +0200, Laurent Hatier
>>> <laurent.hatier@gmail.com> wrote:
>>> > I'm a newbie with Hadoop/MapReduce. I've a problem with hadoop. I set
>>> some
>>> > variables in the run function but when Map running, he can't get the
>>> value
>>> > of theses variables...
>>> > If anyone knows the solution :)
>>>
>>> By the "run function" do you mean the main method that launches the
>>> map/reduce job?  It's no surprise that the mappers (and reducers) won't
>>> know those variables, because they run as completely separate tasks.
>>>
>>> If you're computing something in the setup method for use in the mappers
>>> or reducers you'll have to pass that information along somehow.  If it's
>>> a
>>> String (or something that can easily be made into a String, like an int)
>>> you can set it as a property in the job's Configuration.  For more
>>> complicated data you'll have to serialize it to a file, place the file
>>> into
>>> the distributed cache, and then deserialize the data within the mapper or
>>> reducer's setup method.
>>>
>>> Of course, if the computation is less complicated/time consuming than the
>>> deserialization process, you may as well just recompute the data in each
>>> mapper or reducer.
>>>
>>
>>
>>
>> --
>> Laurent HATIER
>> Étudiant en 2e année du Cycle Ingénieur à l'EISTI
>>
>
>
>
> --
> Laurent HATIER
> Étudiant en 2e année du Cycle Ingénieur à l'EISTI
>



-- 
Laurent HATIER
Étudiant en 2e année du Cycle Ingénieur à l'EISTI

Mime
View raw message