hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hmar...@umbc.edu
Subject Re: specifying number of nodes for job
Date Mon, 08 Sep 2008 20:36:34 GMT
yeah. snickerdoodle. really.


> I see.. so if I have a cluster with n nodes, there is no way for me to
> have
> it spawn on just 2 of those nodes, or just one of those nodes? And
> furthermore, there is no way for me to have it spawn on just a subset of
> the
> processors? Or am I misunderstanding?
>
> Also, when you say "specify the number of tasks for each node" are you
> referring to specifying the number of mappers and reducers I can spawn on
> each node?
>
> -SM
>
> On Sun, Sep 7, 2008 at 8:29 PM, Mafish Liu <mafish@gmail.com> wrote:
>
>> On Mon, Sep 8, 2008 at 2:25 AM, Sandy <snickerdoodle08@gmail.com> wrote:
>>
>> > Hi,
>> >
>> > This may be a silly question, but I'm strangely having trouble finding
>> an
>> > answer for it (perhaps I'm looking in the wrong places?).
>> >
>> > Suppose I have a cluster with n nodes each with m processors.
>> >
>> > I wish to test the performance of, say,  the wordcount program on k
>> > processors, where k is varied from k = 1 ... nm.
>>
>>
>> You can  specify the number of tasks for each node in your
>> hadoop-site.xml
>> file.
>> So you can get k varied from k = n, 2*n....m*n instead of k = 1...nm.
>>
>>
>> > How would I do this? I'm having trouble finding the proper command
>> line
>> > option in the commands manual (
>> > http://hadoop.apache.org/core/docs/current/commands_manual.html)
>> >
>> >
>> >
>> > Thank you very much for you time.
>> >
>> > -SM
>> >
>>
>>
>>
>> --
>> Mafish@gmail.com
>> Institute of Computing Technology, Chinese Academy of Sciences, Beijing.
>>
>



Mime
View raw message