hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rkevinbur...@charter.net
Subject Re: M/R Staticstics
Date Fri, 26 Apr 2013 16:58:04 GMT

I get a message like:

Oops! Google Chrome could not connect to devubuntu05:50030

Where devubuntu05 is the machine (JobTracker, NameNode) running hadoop.

I know it is running because when I do ps I get something like:

kevin@devUbuntu05:~$ ps aux | grep hadoop

hdfs      1095  0.0  2.7 1983656 113048 ?      Sl   Apr23   3:18 
/usr/lib/jvm/java-7-openjdk-amd64//bin/java -Dproc_datanode -Xmx1000m 
-Dhadoop.log.dir=/var/log/hadoop-hdfs 
-Dhadoop.log.file=hadoop-hdfs-datanode-devUbuntu05.log 
-Dhadoop.home.dir=/usr/lib/hadoop -Dhadoop.id.str=hdfs 
-Dhadoop.root.logger=INFO,RFA 
-Djava.library.path=/usr/lib/hadoop/lib/native 
-Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true 
-server -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote -Dhadoop.security.logger=INFO,RFAS 
org.apache.hadoop.hdfs.server.datanode.DataNode

hdfs      1415  0.1  3.1 1972424 126388 ?      Sl   Apr23   5:52 
/usr/lib/jvm/java-7-openjdk-amd64//bin/java -Dproc_namenode -Xmx1000m 
-Dhadoop.log.dir=/var/log/hadoop-hdfs 
-Dhadoop.log.file=hadoop-hdfs-namenode-devUbuntu05.log 
-Dhadoop.home.dir=/usr/lib/hadoop -Dhadoop.id.str=hdfs 
-Dhadoop.root.logger=INFO,RFA 
-Djava.library.path=/usr/lib/hadoop/lib/native 
-Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true 
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote -Dhadoop.security.logger=INFO,RFAS 
org.apache.hadoop.hdfs.server.namenode.NameNode

hdfs      1529  0.0  2.7 1961128 111680 ?      Sl   Apr23   1:49 
/usr/lib/jvm/java-7-openjdk-amd64//bin/java -Dproc_secondarynamenode 
-Xmx1000m -Dhadoop.log.dir=/var/log/hadoop-hdfs 
-Dhadoop.log.file=hadoop-hdfs-secondarynamenode-devUbuntu05.log 
-Dhadoop.home.dir=/usr/lib/hadoop -Dhadoop.id.str=hdfs 
-Dhadoop.root.logger=INFO,RFA 
-Djava.library.path=/usr/lib/hadoop/lib/native 
-Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true 
-Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote 
-Dcom.sun.management.jmxremote -Dhadoop.security.logger=INFO,RFAS 
org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode

yarn      1701  0.0  5.3 1998140 216820 ?      Sl   Apr23   2:36 
/usr/lib/jvm/java-7-openjdk-amd64//bin/java -Dproc_nodemanager -Xmx1000m 
-server -Dhadoop.log.dir=/var/log/hadoop-yarn 
-Dyarn.log.dir=/var/log/hadoop-yarn 
-Dhadoop.log.file=yarn-yarn-nodemanager-devUbuntu05.log 
-Dyarn.log.file=yarn-yarn-nodemanager-devUbuntu05.log 
-Dyarn.home.dir=/usr/lib/hadoop-yarn -Dhadoop.home.dir=/usr/lib/hadoo 
-yarn -Dhadoop.root.logger=INFO,RFA -Dyarn.root.logger=INFO,RFA 
-Djava.library.path=/usr/lib/hadoop/lib/native -classpath 
/etc/hadoop/conf:/etc/hadoop/conf:/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/.//*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-yarn/lib/*:/etc/hadoop/conf/nm-config/log4j.properties

org.apache.hadoop.yarn.server.nodemanager.NodeManager

yarn      1828  0.0  5.3 2128812 217740 ?      Sl   Apr23   2:38 
/usr/lib/jvm/java-7-openjdk-amd64//bin/java -Dproc_resourcemanager 
-Xmx1000m -Dhadoop.log.dir=/var/log/hadoop-yarn 
-Dyarn.log.dir=/var/log/hadoop-yarn 
-Dhadoop.log.file=yarn-yarn-resourcemanager-devUbuntu05.log 
-Dyarn.log.file=yarn-yarn-resourcemanager-devUbuntu05.log 
-Dyarn.home.dir=/usr/lib/hadoop-yarn 
-Dhadoop.home.dir=/usr/lib/hadoop-yarn -Dhadoop.root.logger=INFO,RFA 
-Dyarn.root.logger=INFO,RFA 
-Djava.library.path=/usr/lib/hadoop/lib/native -classpath 
/etc/hadoop/conf:/etc/hadoop/conf:/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-yarn/lib/*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-mapreduce/lib/*:/usr/lib/hadoop-mapreduce/.//*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-yarn/lib/*:/etc/hadoop/conf/rm-config/log4j.properties

org.apache.hadoop.yarn.server.resourcemanager.ResourceManager

So why isn't the JobTracker running?

On Fri, Apr 26, 2013 at 10:37 AM, Rishi Yadav wrote:

  do you see "retired jobs" on job tracker page. There is also "job 
tracker history" on the bottom of page. 

something like this  http://nn.zettabyte.com:50030/jobtracker.jsp 
<http://nn.zettabyte.com:50030/jobtracker.jsp>
Thanks and Regards,
Rishi Yadav



On Fri, Apr 26, 2013 at 7:36 AM, < rkevinburton@charter.net 
<javascript:parent.wgMail.openComposeWindow('rkevinburton@charter.net')> 
> wrote:
When I submit a simple "Hello World" M/R job like WordCount it takes 
less than 5 seconds. The texts show numerous methods for monitoring M/R 
jobs as they are happening but I have yet to see any that show 
statistics about a job after it has completed. Obviously simple jobs 
that take a short amount of time don't allow time to fire up any web 
mage or monitoring tool to see how it progresses through the JobTracker 
and TaskTracker as well as which node it is processed on. Any 
suggestions on how could see this kind of data *after* a job has 
completed?

Mime
View raw message