hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sunil Govind <sunil.gov...@gmail.com>
Subject Re: WebUI's Server don't work and JobHistoryServer missing
Date Thu, 21 Jul 2016 13:05:18 GMT
Hi Mike

yarn.resourcemanager.webapp.address  is to configure Resource Manager Web
UI and you can access ResourceManager Web UI as "http://<host:port>". It
will run on 8088 as default if you are not configuring this property
explicitly. Please refer [
https://hadoop.apache.org/docs/r2.7.2/hadoop-yarn/hadoop-yarn-common/yarn-default.xml]
for more detailed information.

I think you are not running JobHistoryServer. pls refer [
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/ClusterSetup.html]
 "Hadoop Startup" section. You can run history server by "[mapred]$
$HADOOP_PREFIX/sbin/mr-jobhistory-daemon.sh --config $HADOOP_CONF_DIR start
historyserver" command.
If you are interested, you can also run "timelineserver" which is for yarn.

Also you might need to run WebAppProxy server to see the MapReduce job
application master ui. Please refer to above link for cluster setup and it
has detailed information to start various services.

Thanks
Sunil


On Tue, Jul 19, 2016 at 1:59 PM Mike Wenzel <mwenzel@proheris.de> wrote:

> My cluster looks like:
>
>
>
> Node1 - NameNode + ResourceManager
>
> Node2 - SecondaryNameNode
>
> Node3 - DataNode (+NodeManager)
>
> Node4 - DataNode (+NodeManager)
>
> Node5 - DataNode (+NodeManager)
>
>
>
> http://node1:8088/cluster works.
>
>
>
> My problems:
>
> > SecondaryNamenode WebUI: http://node2:50090 doesn’t work (
> ERR_EMPTY_RESPONSE).
>
> > Datanode WebUI: http://node3:50075 looks like there is something wrong:
> https://i.imgur.com/dbTxtkS.png
>
> > Also I’m missing the JobHistory Server. Isn’t the jobhistory server
> included in yarn? I tried accessing some URLs I found on the web declared
> as “default configuration” and always (ERR_EMPTY_RESPONSE).
>
> > When I run a mapreduce job the output still says: “uri to track the job
> is httpd://localhost:8080” and when I try to access any node on this port I
> get no data (ERR_EMPTY_RESPONSE).
>
>
>
> Unfortunately I couldn’t get this problem solved. I only found a few
> guides setting yarn up. I tried them all and my situation only got worse
> when adding more properties on yarn-site.xml. E.g. the cluster
> overview-WebUI http://node1:8088/cluster didn’t worked anymore when
> adding “yarn.resourcemanager.webapp.address” to yarn-site.xml.
>
> Can anyone help me out configuring yarn correctly please?
>
>
>
> My configuration:
>
>
>
> core-site.xml
>
> <?xml version="1.0" encoding="UTF-8"?>
>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>
> <configuration>
>
>   <property>
>
>     <name>hadoop.tmp.dir</name>
>
>     <value>file:///hdfs/tmp</value>
>
>   </property>
>
>   <property>
>
>     <name>fs.defaultFS</name>
>
>     <value>hdfs://node1:54310</value>
>
>   </property>
>
> </configuration>
>
>
>
> hdfs-site.xml
>
> <?xml version="1.0" encoding="UTF-8"?>
>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>
> <configuration>
>
> <!-- NameNode configuration -->
>
>   <property>
>
>     <name>dfs.namenode.name.dir</name>
>
>     <value>file:///hdfs/name</value>
>
>   </property>
>
>   <property>
>
>     <name>dfs.namenode.http-address</name>
>
>     <value>node1:50070</value>
>
>   </property>
>
>   <property>
>
>     <name>dfs.namenode.checkpoint.dir</name>
>
>     <value>file:///hdfs/checkpoint</value>
>
>   </property>
>
>
>
> <!-- SecondaryNameNode configuration -->
>
>   <property>
>
>     <name>dfs.namenode.secondary.http-address</name>
>
>     <value>node2:50090</value>
>
>   </property>
>
>
>
> <!-- DataNode configuration -->
>
>   <property>
>
>     <name>dfs.datanode.data.dir</name>
>
>     <value>file:///hdfs/data</value>
>
>   </property>
>
> </configuration>
>
>
>
> yarn-site.xml
>
> <?xml version="1.0"?>
>
> <configuration>
>
> <!-- Site specific YARN configuration properties -->
>
> <!-- RessourceManager configuration -->
>
>   <property>
>
>     <name>yarn.resourcemanager.hostname</name>
>
>     <value>node1</value>
>
>   </property>
>
>   <property>
>
>     <name>yarn.resourcemanager.resource-tracker.address</name>
>
>     <value>node1:8031</value>
>
>   </property>
>
>   <property>
>
>     <name>yarn.resourcemanager.scheduler.address</name>
>
>     <value>node1:8030</value>
>
>   </property>
>
>   <property>
>
>     <name>yarn.resourcemanager.address</name>
>
>     <value>node1:8032</value>
>
>   </property>
>
> </configuration>
>
>
>
> masters
>
> node2
>
>
>
> slaves
>
> node3
>
> node4
>
> node5
>

Mime
View raw message