hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gurmukh Singh <gurmukh.dhil...@yahoo.com.INVALID>
Subject Re: how to set a rpc timeout on yarn application.
Date Sat, 30 Sep 2017 23:59:01 GMT
Hi

Can you explain me the job a bit, there are few rpc timeout like at 
datanode level, mapper timeouts etc


On 28/9/17 1:47 pm, Demon King wrote:
> Hi,
>      We have finished a yarn application and deploy it to hadoop 2.6.0 
> cluster. But if one machine in cluster is down. Our application will 
> hang on NMClientAsyncImpl.stop(). The last log is:
>
>  953851 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : a14k17307.em21.tbsite.net:8041 
> <http://a14k17307.em21.tbsite.net:8041/>
>  953852 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : a24a21449.em21.tbsite.net:8041 
> <http://a24a21449.em21.tbsite.net:8041/>
>  953853 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : e92e09611.em21.tbsite.net:8041 
> <http://e92e09611.em21.tbsite.net:8041/>
>  953854 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : a24a21477.em21.tbsite.net:8041 
> <http://a24a21477.em21.tbsite.net:8041/>
>  953855 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : e92e09590.em21.tbsite.net:8041 
> <http://e92e09590.em21.tbsite.net:8041/>
>  953856 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : a14k17314.em21.tbsite.net:8041 
> <http://a14k17314.em21.tbsite.net:8041/>
>  953857 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : a24a21474.em21.tbsite.net:8041 
> <http://a24a21474.em21.tbsite.net:8041/>
>  953858 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : e92e09574.em21.tbsite.net:8041 
> <http://e92e09574.em21.tbsite.net:8041/>
>  953859 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : e92e09579.em21.tbsite.net:8041 
> <http://e92e09579.em21.tbsite.net:8041/>
>  953860 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : a14k17332.em21.tbsite.net:8041 
> <http://a14k17332.em21.tbsite.net:8041/>
>  953861 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : a14k17300.em21.tbsite.net:8041 
> <http://a14k17300.em21.tbsite.net:8041/>
>  953862 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : e93g17455.em21.tbsite.net:8041 
> <http://e93g17455.em21.tbsite.net:8041/>
>  953863 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : e92e09595.em21.tbsite.net:8041 
> <http://e92e09595.em21.tbsite.net:8041/>
>  953864 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : a14k17309.em21.tbsite.net:8041 
> <http://a14k17309.em21.tbsite.net:8041/>
>  953865 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : a14k17294.em21.tbsite.net:8041 
> <http://a14k17294.em21.tbsite.net:8041/>
>  953866 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : a14k17316.em21.tbsite.net:8041 
> <http://a14k17316.em21.tbsite.net:8041/>
>  953867 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : e92e09592.em21.tbsite.net:8041 
> <http://e92e09592.em21.tbsite.net:8041/>
>  953868 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : a24a21472.em21.tbsite.net:8041 
> <http://a24a21472.em21.tbsite.net:8041/>
>  953869 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : a14k17337.em21.tbsite.net:8041 
> <http://a14k17337.em21.tbsite.net:8041/>
>  953870 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : e92e09580.em21.tbsite.net:8041 
> <http://e92e09580.em21.tbsite.net:8041/>
>  953871 17/09/27 10:46:52 INFO impl.ContainerManagementProtocolProxy: 
> Opening proxy : e92e09619.em21.tbsite.net:8041 
> <http://e92e09619.em21.tbsite.net:8041/>
>
> Is any way to set a timeout for NMClientAsyncImpl when call it stop?
>
> we init it by :
>        NMClientAsync nmClient = 
> NMClientAsync.createNMClientAsync(nmCallbackHandler);
>
> and then :
>
>         yarnConfig = new YarnConfiguration();
> nmClient .init(yarnConfig);
>         nmClient .start();
> our code not modify nmClient.


Mime
View raw message