hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Lilley <john.lil...@redpoint.net>
Subject RE: very long timeout on failed RM connect
Date Mon, 10 Feb 2014 19:11:35 GMT
I tried:
conf.set("yarn.resourcemanager.connect.max-wait.ms", "10000");
conf.set("yarn.resourcemanager.connect.retry-interval.ms", "1000");

But it has no apparent effect.  Still hangs for a very long time.

From: Jian He [mailto:jhe@hortonworks.com]
Sent: Monday, February 10, 2014 11:05 AM
To: user@hadoop.apache.org
Subject: Re: very long timeout on failed RM connect

Setting the following two properties may solve your problem.

controls Maximum time to wait to establish connection to ResourceManager.

controls How often to try connecting to the ResourceManager.


On Mon, Feb 10, 2014 at 6:44 AM, John Lilley <john.lilley@redpoint.net<mailto:john.lilley@redpoint.net>>
Our application (running outside the Hadoop cluster) connects to the RM through YarnClient.
 This works fine, except we've found that if the RM address or port is misconfigured in our
software, or a firewall blocks access, the first call into the client (in this case getNodeReports)
hangs for a very long time.  I've tried
                conf.set("ipc.client.connect.max.retries", "2");
But this doesn't help.  Is there a configuration setting I can make on the YarnClient that
will reduce this hang time?
I understand why this long-winded retry strategy exists, in order to prevent a highly-loaded
cluster from failing jobs.  But it is not appropriate for an interactive application.

NOTICE: This message is intended for the use of the individual or entity to which it is addressed
and may contain information that is confidential, privileged and exempt from disclosure under
applicable law. If the reader of this message is not the intended recipient, you are hereby
notified that any printing, copying, dissemination, distribution, disclosure or forwarding
of this communication is strictly prohibited. If you have received this communication in error,
please contact the sender immediately and delete it from your system. Thank You.

View raw message