kylin-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "op" <520075...@qq.com>
Subject 回复: yarn configuration problem when building kylin
Date Fri, 13 Oct 2017 09:21:02 GMT
hello ShaoFeng.
the situation above is ,when building a cube,the first two step can successfully finish,but
will Get stuck at the third step ,the log ptinting Retrying connect to server: 0.0.0.0/0.0.0.0:8032.
Already tried x times without stop。







------------------ 原始邮件 ------------------
发件人: "ShaoFeng Shi";<shaofengshi@apache.org>;
发送时间: 2017年10月13日(星期五) 下午5:02
收件人: "user"<user@kylin.apache.org>;

主题: Re: yarn configuration problem when building kylin



Obviously, Kylin didn't aware your yarn-site.xml, causing it connecting with a non-existing
address. 

Please check whether the right yarn-site.xml is in the Hadoop configuration folder, e.g, /etc/hadoop/conf.
You can also try to run a sample Hadoop job from the Kylin node, to verify whether the node
is properly configured.


BTW, English is the recommended language for communication, because Kylin users are from different
countries. 


2017-10-13 15:13 GMT+08:00 op <520075694@qq.com>:
hi,Shuangyin Ge
our clusters contains 63 datanodes ,resourcemanager and namenode are set up in the same 2
nodes ,both enabled HA..they are working stably for some years.  do you think we have to change
some configurations?
we put kylin in client node 129 and resourcemanagers  are in 225 and 236


in addition,can you speak chinese?


thanks
------------------ 原始邮件 ------------------
发件人: "Shuangyin Ge";<gosoy.ge@gmail.com>;
发送时间: 2017年10月13日(星期五) 下午3:03
收件人: "user"<user@kylin.apache.org>;

主题: Re: yarn configuration problem when building kylin



Hello op,


Can you try to specify yarn.resourcemanager.hostname.rm1 and yarn.resourcemanager.hostname.rm2
in yarn-site.xml as well following https://hadoop.apache.org/docs/r2.8.0/hadoop-yarn/hadoop-yarn-site/ResourceManagerHA.html?

2017-10-13 14:44 GMT+08:00 op <520075694@qq.com>:
when i builing my cube,the progress is always pending,then i find this in kylin.log,can't
connect to the correct resourcemanager address,i've checked my environment,can you give me
some advice?


2017-10-13 14:33:48,978 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] client.RMProxy:56
: Connecting to ResourceManager at /0.0.0.0:8032
2017-10-13 14:33:50,061 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 0 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)
2017-10-13 14:33:51,062 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 1 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)
2017-10-13 14:33:52,063 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 2 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)
2017-10-13 14:33:53,064 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 3 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)
2017-10-13 14:33:54,065 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 4 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)
2017-10-13 14:33:55,067 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 5 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)
2017-10-13 14:33:56,068 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 6 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)
2017-10-13 14:33:57,069 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 7 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)
2017-10-13 14:33:58,070 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 8 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)
2017-10-13 14:33:59,071 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 9 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)
2017-10-13 14:34:00,072 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 10 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)
2017-10-13 14:34:01,073 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 11 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)
2017-10-13 14:34:02,074 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 12 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)
2017-10-13 14:34:03,075 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 13 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)
2017-10-13 14:34:04,076 INFO  [Job a8f48457-7c00-4cae-8857-c6e61c10213d-63] ipc.Client:783
: Retrying connect to server: 0.0.0.0/0.0.0.0:8032. Already tried 14 time(s); retry policy
is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1 SECONDS)



my yarn enabled HA,there are some of the configurations:


<property>
   <name>yarn.resourcemanager.cluster-id</name>
   <value>boh</value>
   <final>false</final>
</property>  


<property>
   <name>yarn.resourcemanager.ha.rm-ids</name>
   <value>rm1,rm2</value>
   <final>false</final>
</property>


<property>
   <name>yarn.resourcemanager.webapp.address.rm1</name>
   <value>hadoop001:23188</value>
   <final>false</final>
</property>


<property>
   <name>yarn.resourcemanager.webapp.https.address.rm1</name>
   <value>hadoop001:23189</value>
   <final>false</final>
</property>


<property>
   <name>yarn.resourcemanager.resource-tracker.address.rm1</name>
   <value>hadoop001:23125</value>
   <final>false</final>
</property>


<property>
   <name>yarn.resourcemanager.scheduler.address.rm1</name>
   <value>hadoop001:23130</value>
   <final>false</final>
</property>


<property>
   <name>yarn.resourcemanager.address.rm1</name>
   <value>hadoop001:23140</value>
   <final>false</final>
</property>


<property>
   <name>yarn.resourcemanager.admin.address.rm1</name>
   <value>hadoop001:23141</value>
   <final>false</final>
</property>


<property>
   <name>yarn.resourcemanager.webapp.address.rm2</name>
   <value>hadoop011:23188</value>
   <final>false</final>
</property>


<property>
   <name>yarn.resourcemanager.webapp.https.address.rm2</name>
   <value>hadoop011:23189</value>
   <final>false</final>
</property>


<property>
   <name>yarn.resourcemanager.resource-tracker.address.rm2</name>
   <value>hadoop011:23125</value>
   <final>false</final>
</property>


<property>
   <name>yarn.resourcemanager.scheduler.address.rm2</name>
   <value>hadoop011:23130</value>
   <final>false</final>
</property>


<property>
   <name>yarn.resourcemanager.address.rm2</name>
   <value>hadoop011:23140</value>
   <final>false</final>
</property>


<property>
   <name>yarn.resourcemanager.admin.address.rm2</name>
   <value>hadoop011:23141</value>
   <final>false</final>
</property>












-- 
Best regards,

Shaofeng Shi 史少锋
Mime
View raw message