kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ShaoFeng Shi <shaofeng...@apache.org>
Subject Re: yarn configuration problem when building kylin
Date Sun, 15 Oct 2017 01:24:12 GMT
Kylin support YARN HA since a very early version. And many users are
running in this way.

If you can provide the Hadoop version and full yarn-site.xml, that would be
easier for investigation.

2017-10-14 16:24 GMT+08:00 op <520075694@qq.com>:

> doesn’t apache-kylin-2.0.0-bin-hbase098.tar.gz support YARN HA? i just
> changed my yarn-site.xml,disabled YARN HA,and then the resorcemanager can
> be successfully detected。。
>
> ------------------ 原始邮件 ------------------
> *发件人:* "╰╮爱ャ国灬";<520075694@qq.com>;
> *发送时间:* 2017年10月13日(星期五) 下午5:21
> *收件人:* "user"<user@kylin.apache.org>;
> *主题:* 回复: yarn configuration problem when building kylin
>
> hello ShaoFeng.
> the situation above is ,when building a cube,the first two step can
> successfully finish,but will Get stuck at the third step ,the log ptinting
> Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried x times
> without stop。
>
>
>
> ------------------ 原始邮件 ------------------
> *发件人:* "ShaoFeng Shi";<shaofengshi@apache.org>;
> *发送时间:* 2017年10月13日(星期五) 下午5:02
> *收件人:* "user"<user@kylin.apache.org>;
> *主题:* Re: yarn configuration problem when building kylin
>
> Obviously, Kylin didn't aware your yarn-site.xml, causing it connecting
> with a non-existing address.
>
> Please check whether the right yarn-site.xml is in the Hadoop
> configuration folder, e.g, /etc/hadoop/conf. You can also try to run a
> sample Hadoop job from the Kylin node, to verify whether the node is
> properly configured.
>
> BTW, English is the recommended language for communication, because Kylin
> users are from different countries.
>
> 2017-10-13 15:13 GMT+08:00 op <520075694@qq.com>:
>
>> hi,Shuangyin Ge
>> our clusters contains 63 datanodes ,resourcemanager and namenode are set
>> up in the same 2 nodes ,both enabled HA..they are working stably for some
>> years.  do you think we have to change some configurations?
>> we put kylin in client node 129 and resourcemanagers  are in 225 and 236
>>
>> in addition,can you speak chinese?
>>
>> thanks
>> ------------------ 原始邮件 ------------------
>> *发件人:* "Shuangyin Ge";<gosoy.ge@gmail.com>;
>> *发送时间:* 2017年10月13日(星期五) 下午3:03
>> *收件人:* "user"<user@kylin.apache.org>;
>> *主题:* Re: yarn configuration problem when building kylin
>>
>> Hello op,
>>
>> Can you try to specify yarn.resourcemanager.hostname.rm1 and
>> yarn.resourcemanager.hostname.rm2 in yarn-site.xml as well following
>> https://hadoop.apache.org/docs/r2.8.0/hadoop-yarn/
>> hadoop-yarn-site/ResourceManagerHA.html?
>>
>> 2017-10-13 14:44 GMT+08:00 op <520075694@qq.com>:
>>
>>> when i builing my cube,the progress is always pending,then i find this
>>> in kylin.log,can't connect to the correct resourcemanager address,i've
>>> checked my environment,can you give me some advice?
>>>
>>> 2017-10-13 14:33:48,978 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> client.RMProxy:56 : Connecting to ResourceManager at /0.0.0.0:8032
>>> 2017-10-13 14:33:50,061 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>> 2017-10-13 14:33:51,062 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>> 2017-10-13 14:33:52,063 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>> 2017-10-13 14:33:53,064 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>> 2017-10-13 14:33:54,065 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>> 2017-10-13 14:33:55,067 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>> 2017-10-13 14:33:56,068 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>> 2017-10-13 14:33:57,069 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>> 2017-10-13 14:33:58,070 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>> 2017-10-13 14:33:59,071 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>> 2017-10-13 14:34:00,072 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 10 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>> 2017-10-13 14:34:01,073 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 11 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>> 2017-10-13 14:34:02,074 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 12 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>> 2017-10-13 14:34:03,075 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 13 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>> 2017-10-13 14:34:04,076 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63]
>>> ipc.Client:783 : Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
>>> Already tried 14 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
>>> sleepTime=1 SECONDS)
>>>
>>> my yarn enabled HA,there are some of the configurations:
>>>
>>> <property>
>>>    <name>yarn.resourcemanager.cluster-id</name>
>>>    <value>boh</value>
>>>    <final>false</final>
>>> </property>
>>>
>>> <property>
>>>    <name>yarn.resourcemanager.ha.rm-ids</name>
>>>    <value>rm1,rm2</value>
>>>    <final>false</final>
>>> </property>
>>>
>>> <property>
>>>    <name>yarn.resourcemanager.webapp.address.rm1</name>
>>>    <value>hadoop001:23188</value>
>>>    <final>false</final>
>>> </property>
>>>
>>> <property>
>>>    <name>yarn.resourcemanager.webapp.https.address.rm1</name>
>>>    <value>hadoop001:23189</value>
>>>    <final>false</final>
>>> </property>
>>>
>>> <property>
>>>    <name>yarn.resourcemanager.resource-tracker.address.rm1</name>
>>>    <value>hadoop001:23125</value>
>>>    <final>false</final>
>>> </property>
>>>
>>> <property>
>>>    <name>yarn.resourcemanager.scheduler.address.rm1</name>
>>>    <value>hadoop001:23130</value>
>>>    <final>false</final>
>>> </property>
>>>
>>> <property>
>>>    <name>yarn.resourcemanager.address.rm1</name>
>>>    <value>hadoop001:23140</value>
>>>    <final>false</final>
>>> </property>
>>>
>>> <property>
>>>    <name>yarn.resourcemanager.admin.address.rm1</name>
>>>    <value>hadoop001:23141</value>
>>>    <final>false</final>
>>> </property>
>>>
>>> <property>
>>>    <name>yarn.resourcemanager.webapp.address.rm2</name>
>>>    <value>hadoop011:23188</value>
>>>    <final>false</final>
>>> </property>
>>>
>>> <property>
>>>    <name>yarn.resourcemanager.webapp.https.address.rm2</name>
>>>    <value>hadoop011:23189</value>
>>>    <final>false</final>
>>> </property>
>>>
>>> <property>
>>>    <name>yarn.resourcemanager.resource-tracker.address.rm2</name>
>>>    <value>hadoop011:23125</value>
>>>    <final>false</final>
>>> </property>
>>>
>>> <property>
>>>    <name>yarn.resourcemanager.scheduler.address.rm2</name>
>>>    <value>hadoop011:23130</value>
>>>    <final>false</final>
>>> </property>
>>>
>>> <property>
>>>    <name>yarn.resourcemanager.address.rm2</name>
>>>    <value>hadoop011:23140</value>
>>>    <final>false</final>
>>> </property>
>>>
>>> <property>
>>>    <name>yarn.resourcemanager.admin.address.rm2</name>
>>>    <value>hadoop011:23141</value>
>>>    <final>false</final>
>>> </property>
>>>
>>
>>
>
>
> --
> Best regards,
>
> Shaofeng Shi 史少锋
>
>


-- 
Best regards,

Shaofeng Shi 史少锋

Mime
View raw message