hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bertrand Dechoux <decho...@gmail.com>
Subject Re: What is the preferred way to pass a small number of configuration parameters to a mapper or reducer
Date Mon, 31 Dec 2012 08:17:31 GMT
*G*. Use cascading so that way you don't have to actually provide the
parameters yourself because there is a transparent serialization of what
will become the mapper and the reducer. (but it is really a hidden kind-of *
A*).

http://www.cascading.org/

About certifications, of course, cheating is not allowed. And if you are
indeed cheating, you are open the 'retributions' you agreed on.
But at the same time, you can find online resources which are plain wrong.
I think the only good answers are
*1)* use the API and figure it yourself
*2)* Do not trust everybody (even well intentioned people can be wrong and
the same can be said about public opinion)
*3)* read a good reference (like http://hadoopbook.com/)

The mailing list could have a rule stating the such post are not allowed.
It really looks like a copy-and-paste from somewhere. Any author should
provide more context if there really is a point which is not understood.

Bertrand


On Sun, Dec 30, 2012 at 8:38 PM, Niels Basjes <Niels@basjes.nl> wrote:

> F. put a mongodb replica set on all hadoop workernodes and let the tasks
> query the mongodb at localhost.
>
> (this is what I did recently with a multi GiB dataset)
>
> --
> Met vriendelijke groet,
> Niels Basjes
> (Verstuurd vanaf mobiel )
> Op 30 dec. 2012 20:01 schreef "Jonathan Bishop" <jbishop.rwc@gmail.com>
> het volgende:
>
> E. Store them in hbase...
>>
>>
>> On Sun, Dec 30, 2012 at 12:24 AM, Hemanth Yamijala <
>> yhemanth@thoughtworks.com> wrote:
>>
>>> If it is a small number, A seems the best way to me.
>>>
>>> On Friday, December 28, 2012, Kshiva Kps wrote:
>>>
>>>>
>>>> Which one is current ..
>>>>
>>>>
>>>> What is the preferred way to pass a small number of configuration
>>>> parameters to a mapper or reducer?
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *A.  *As key-value pairs in the jobconf object.
>>>>
>>>> * *
>>>>
>>>> *B.  *As a custom input key-value pair passed to each mapper or
>>>> reducer.
>>>>
>>>> * *
>>>>
>>>> *C.  *Using a plain text file via the Distributedcache, which each
>>>> mapper or reducer reads.
>>>>
>>>> * *
>>>>
>>>> *D.  *Through a static variable in the MapReduce driver class (i.e.,
>>>> the class that submits the MapReduce job).
>>>>
>>>>
>>>>
>>>> *Answer: B*
>>>>
>>>>
>>>>
>>>
>>

Mime
View raw message