hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Bond <john.r.b...@gmail.com>
Subject Re: How to get number of live nodes in hadoop
Date Tue, 11 Oct 2011 18:26:06 GMT
for the nmap fans give the hadoop-jobtracker-info script from the
below link ago ;)

https://github.com/b4ldr/nse-scripts

disclaimer: this email is about shameless, self promotion

On 11 October 2011 19:52, patrick sang <silvianhadoop@gmail.com> wrote:
> for cli fans,
>
> In DFS side - ./bin/hadoop dfsadmin -report
> In MR side - ./bin/hadoop job -list-active-trackers
>
> hth,
> P
>
> On Tue, Oct 11, 2011 at 5:27 AM, Uma Maheswara Rao G 72686 <
> maheswara@huawei.com> wrote:
>
>> Hello Raimon,
>>
>> In DFS to know the DN status you can use getDataNodeStats API from
>> Distributed fileSystem.
>>
>> In MR, to know the number of active trackers, you can use getClusterStatus
>> from jobclient. It will give other stats as well.
>>
>> Hope this will help.
>>
>> Regards,
>> Uma
>>
>>
>> ----- Original Message -----
>> From: Raimon Bosch <raimon.bosch@gmail.com>
>> Date: Tuesday, October 11, 2011 5:48 pm
>> Subject: How to get number of live nodes in hadoop
>> To: common-user@hadoop.apache.org
>>
>> > Hi,
>> >
>> > Following this instructions at
>> > http://wiki.apache.org/hadoop/HowManyMapsAndReduces I've read that
>> > the best
>> > amount of reducers for one process is 0.95 or 1.75 * (nodes *
>> > mapred.tasktracker.tasks.maximum) so I would like to call
>> > to conf.setNumReduceTasks(int num) according to how many nodes I have
>> > working.
>> >
>> > So how can I get the number of live nodes from my hadoop code?
>> >
>> > Thanks in advance,
>> > Raimon Bosch.
>> >
>>
>

Mime
View raw message