hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From patrick sang <silvianhad...@gmail.com>
Subject Re: How to get number of live nodes in hadoop
Date Tue, 11 Oct 2011 17:52:55 GMT
for cli fans,

In DFS side - ./bin/hadoop dfsadmin -report
In MR side - ./bin/hadoop job -list-active-trackers

hth,
P

On Tue, Oct 11, 2011 at 5:27 AM, Uma Maheswara Rao G 72686 <
maheswara@huawei.com> wrote:

> Hello Raimon,
>
> In DFS to know the DN status you can use getDataNodeStats API from
> Distributed fileSystem.
>
> In MR, to know the number of active trackers, you can use getClusterStatus
> from jobclient. It will give other stats as well.
>
> Hope this will help.
>
> Regards,
> Uma
>
>
> ----- Original Message -----
> From: Raimon Bosch <raimon.bosch@gmail.com>
> Date: Tuesday, October 11, 2011 5:48 pm
> Subject: How to get number of live nodes in hadoop
> To: common-user@hadoop.apache.org
>
> > Hi,
> >
> > Following this instructions at
> > http://wiki.apache.org/hadoop/HowManyMapsAndReduces I've read that
> > the best
> > amount of reducers for one process is 0.95 or 1.75 * (nodes *
> > mapred.tasktracker.tasks.maximum) so I would like to call
> > to conf.setNumReduceTasks(int num) according to how many nodes I have
> > working.
> >
> > So how can I get the number of live nodes from my hadoop code?
> >
> > Thanks in advance,
> > Raimon Bosch.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message