hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Margusja <mar...@roo.ee>
Subject Re: ApplicationMaster link on cluster web page does not work
Date Thu, 28 Aug 2014 19:34:23 GMT
Moved resourcemanager to another server and it works. I guess I have 
some network miss routing there :)

Best regards, Margus (Margusja) Roo
+372 51 48 780
http://margus.roo.ee
http://ee.linkedin.com/in/margusroo
skype: margusja
ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"

On 28/08/14 21:39, Margusja wrote:
> More information
>
> after I started resourcemanager
> [root@vm38 ~]# /etc/init.d/hadoop-yarn-resourcemanager start
> Starting Hadoop resourcemanager:                           [  OK ]
>
> and I open cluster web interface there is some tcp connections to 8088:
>
> [root@vm38 ~]# netstat -np | grep 8088
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:84.50.21.39:61120    ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:84.50.21.39:64412    ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:84.50.21.39:50139    ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:84.50.21.39:64407    ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:84.50.21.39:58817    ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:84.50.21.39:52250    ESTABLISHED 15723/java
>
> this is ok
>
> Now I started new map reduce job and got tracking url 
> http://server:8088/proxy/application_1409250808355_0001/
>
> now there are loads of connections and tracking url does not response:
> tcp        0      0 ::ffff:90.190.106.33:46910 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:35984 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:37559  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:44154 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:45417 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:57294 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:47949  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:55330 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp      432      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:45467  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:58405 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:55580 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:51992  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:44578 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:38686  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:39242 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:55916 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:48064 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:35148 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:42638 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:50836 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:44789  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:55051 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:39207  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:53781 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:54180 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:56344 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:52707 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:52274 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:45417  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:46627  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:44583  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:40928 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:47014 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:60939 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:52274  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:38391 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:47321  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:36186  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:39160 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:54352 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:59425  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:46627 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:57140  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:48788  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:37559 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:55052  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:47104  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:47515 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:45655  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:48064  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:45467 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:47343  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:42678 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:33116  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:40077 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:49100 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:44547  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:40050  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:43073 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:38686 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:41480 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:56344  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:42933 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:45400  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:33023 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:42678  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:43331 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:48524  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:51789 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:44447  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:35984  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:50472 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:55326 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:44356 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:48788 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:44560  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:38690  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:57434  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:45282 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:38284 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:44411 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:55712 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:48929 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:49288 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:42505 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:38391  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:54537  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:43331  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:47141  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:60823 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:55712  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:42313  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:45879  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:33412 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:51992 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:37513  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:39260 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:8088 
> ::ffff:90.190.106.33:46269  ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:46087 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> tcp        0      0 ::ffff:90.190.106.33:47321 
> ::ffff:90.190.106.33:8088   ESTABLISHED 15723/java
> ...
> ...
> ...
>
> wtf?
>
> Best regards, Margus (Margusja) Roo
> +372 51 48 780
> http://margus.roo.ee
> http://ee.linkedin.com/in/margusroo
> skype: margusja
> ldapsearch -x -h ldap.sk.ee -b c=EE "(serialNumber=37303140314)"
>
> On 28/08/14 20:40, Margusja wrote:
>> Hi
>>
>> <configuration>
>>
>> <!-- Site specific YARN configuration properties -->
>> <property>
>>   <name>yarn.app.mapreduce.am.staging-dir</name>
>>   <value>/user</value>
>> </property>
>>
>> <property>
>>     <name>yarn.nodemanager.aux-services</name>
>>     <value>mapreduce_shuffle</value>
>> </property>
>>
>> <property>
>> <name>yarn.nodemanager.aux-services.mapreduce_shuffle.class</name>
>> <value>org.apache.hadoop.mapred.ShuffleHandler</value>
>> </property>
>>
>> <property>
>>         <name>yarn.application.classpath</name>
>>         <value>
>> /etc/hadoop/conf,/usr/lib/hadoop/*,/usr/lib/hadoop/lib/*,/usr/lib/hadoop-hdfs/*,/usr/lib/hadoop-hdfs/lib/*,/usr/lib/hadoop-yarn/*,/usr/lib/hadoop-yarn/lib/*,/usr/lib/hadoop-mapreduce/*,/usr/lib/hadoop-mapreduce/lib/*,/home/hduser/mahout-1.0-snapshot/math/target/*

>>
>>         </value>
>> </property>
>>
>> <property>
>> <name>yarn.resourcemanager.scheduler.address</name>
>>         <value>0.0.0.0:8030</value>
>> </property>
>>
>> <property>
>>         <name>yarn.web-proxy.address</name>
>>         <value>vm38.dbweb.ee:8089</value>
>> </property>
>>
>> </configuration>
>>
>> when i click job's ApplicationMaster link 
>> (http://vm38.dbweb.ee:8089/proxy/application_1409246753441_0001/) on 
>> claster all applications list I will get:
>>
>>
>>    HTTP ERROR 404
>>
>> Problem accessing /proxy/application_1409246753441_0001/mapreduce. 
>> Reason:
>>
>>     NOT_FOUND
>>
>> ------------------------------------------------------------------------
>> /Powered by Jetty://
>>
>> /I use yarn.web-proxy because without yarn.web-proxy when I click 
>> ApplicationMaster link I will have loads of tcp connections with 
>> established statuses.
>> In yarn-proxy log there is nothing interesting
>>
>


Mime
View raw message