hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Uma Maheswara Rao G 72686 <mahesw...@huawei.com>
Subject Re: How to get number of live nodes in hadoop
Date Tue, 11 Oct 2011 12:27:04 GMT
Hello Raimon,

In DFS to know the DN status you can use getDataNodeStats API from Distributed fileSystem.

In MR, to know the number of active trackers, you can use getClusterStatus from jobclient.
It will give other stats as well.

Hope this will help.


----- Original Message -----
From: Raimon Bosch <raimon.bosch@gmail.com>
Date: Tuesday, October 11, 2011 5:48 pm
Subject: How to get number of live nodes in hadoop
To: common-user@hadoop.apache.org

> Hi,
> Following this instructions at
> http://wiki.apache.org/hadoop/HowManyMapsAndReduces I've read that 
> the best
> amount of reducers for one process is 0.95 or 1.75 * (nodes *
> mapred.tasktracker.tasks.maximum) so I would like to call
> to conf.setNumReduceTasks(int num) according to how many nodes I have
> working.
> So how can I get the number of live nodes from my hadoop code?
> Thanks in advance,
> Raimon Bosch.

View raw message