Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of maheswara@huawei.com
 designates 206.16.17.211 as permitted sender)
Date: Tue, 11 Oct 2011 17:27:04 +0500
From: Uma Maheswara Rao G 72686 <maheswara@huawei.com>
Subject: Re: How to get number of live nodes in hadoop
In-reply-to: 
 <CANvqUdUXE2q4DXqNMmg0_WC=oKrrNzy8srUH+GD33NepqV9XxQ@mail.gmail.com>
To: common-user@hadoop.apache.org
Message-id: <fdd49b622976.2976fdd49b62@huawei.com>
MIME-version: 1.0
Content-type: text/plain; charset=us-ascii
Content-language: en
Content-transfer-encoding: 7BIT
Content-disposition: inline
Priority: normal
References: 
 <CANvqUdUXE2q4DXqNMmg0_WC=oKrrNzy8srUH+GD33NepqV9XxQ@mail.gmail.com>

Hello Raimon,

In DFS to know the DN status you can use getDataNodeStats API from Distributed fileSystem.

In MR, to know the number of active trackers, you can use getClusterStatus from jobclient. It will give other stats as well.

Hope this will help.

Regards,
Uma


----- Original Message -----
From: Raimon Bosch <raimon.bosch@gmail.com>
Date: Tuesday, October 11, 2011 5:48 pm
Subject: How to get number of live nodes in hadoop
To: common-user@hadoop.apache.org

> Hi,
> 
> Following this instructions at
> http://wiki.apache.org/hadoop/HowManyMapsAndReduces I've read that 
> the best
> amount of reducers for one process is 0.95 or 1.75 * (nodes *
> mapred.tasktracker.tasks.maximum) so I would like to call
> to conf.setNumReduceTasks(int num) according to how many nodes I have
> working.
> 
> So how can I get the number of live nodes from my hadoop code?
> 
> Thanks in advance,
> Raimon Bosch.
>