Return-Path: X-Original-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-hdfs-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E6D361184C for ; Thu, 10 Jul 2014 15:39:44 +0000 (UTC) Received: (qmail 3810 invoked by uid 500); 10 Jul 2014 15:39:39 -0000 Delivered-To: apmail-hadoop-hdfs-user-archive@hadoop.apache.org Received: (qmail 3658 invoked by uid 500); 10 Jul 2014 15:39:39 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 3642 invoked by uid 99); 10 Jul 2014 15:39:39 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jul 2014 15:39:39 +0000 X-ASF-Spam-Status: No, hits=2.9 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [68.178.252.235] (HELO p3plsmtpa12-06.prod.phx3.secureserver.net) (68.178.252.235) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 10 Jul 2014 15:39:33 +0000 Received: from [10.0.1.13] ([50.134.136.216]) by p3plsmtpa12-06.prod.phx3.secureserver.net with id QffA1o00i4gJeJS01ffBRC; Thu, 10 Jul 2014 08:39:12 -0700 From: Geoff Thompson Content-Type: multipart/alternative; boundary="Apple-Mail=_0A7A5F58-10D4-42CE-86B7-787DB0C06060" Message-Id: Mime-Version: 1.0 (Mac OS X Mail 7.3 \(1878.2\)) Subject: Re: how to access configuration properties on a remote Hadoop cluster Date: Thu, 10 Jul 2014 09:39:09 -0600 References:

To: user@hadoop.apache.org In-Reply-To: X-Mailer: Apple Mail (2.1878.2) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_0A7A5F58-10D4-42CE-86B7-787DB0C06060 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=us-ascii Hi Adam, Thanks for the suggestion. Geoff On Jul 9, 2014, at 2:02 PM, Adam Kawa wrote: > Instead of /conf, If you have = application id and job id, you can query the Resource Manager for the = configuration of this particular application. You can use HTTP and Java = API for that. >=20 >=20 >=20 > 2014-07-09 21:42 GMT+02:00 Geoff Thompson : > Hello, >=20 > Is there a way to query the Resource Manager for configuration = properties from an external client process other than using the web = interface? >=20 > Our background: We run a YARN application by running a Client on an = external machine that may access one of many remote Hadoop clusters. The = configuration properties on the Client machine are completely = independent of the configuration properties on the remote cluster. >=20 > Currently I am getting configuration properties by using HTTP to get = /conf. However, if possible, I would = prefer some form of Hadoop interface. I have looked through YarnClient = and YarnClientApplication and I have not discovered anything that = enables asking the Resource Manager about properties. >=20 > Does anyone have any suggestions? >=20 > Thanks, >=20 > Geoff >=20 >=20 >=20 --Apple-Mail=_0A7A5F58-10D4-42CE-86B7-787DB0C06060 Content-Transfer-Encoding: 7bit Content-Type: text/html; charset=us-ascii Hi Adam,

Thanks for the suggestion.

Geoff

On Jul 9, 2014, at 2:02 PM, Adam Kawa <kawa.adam@gmail.com> wrote:

Instead of <Resource-Manager-WebApp-Address>/conf, If you have application id and job id, you can query the Resource Manager for the configuration of this particular application. You can use HTTP and Java API for that.

2014-07-09 21:42 GMT+02:00 Geoff Thompson <geoff@bearpeak.com>:

Hello,

Is there a way to query the Resource Manager for configuration properties from an external client process other than using the web interface?

Our background: We run a YARN application by running a Client on an external machine that may access one of many remote Hadoop clusters. The configuration properties on the Client machine are completely independent of the configuration properties on the remote cluster.

Currently I am getting configuration properties by using HTTP to get <Resource-Manager-WebApp-Address>/conf. However, if possible, I would prefer some form of Hadoop interface. I have looked through YarnClient and YarnClientApplication and I have not discovered anything that enables asking the Resource Manager about properties.

Does anyone have any suggestions?

Thanks,

Geoff

--Apple-Mail=_0A7A5F58-10D4-42CE-86B7-787DB0C06060--