hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anupam Seth (Updated) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-3843) Job summary log file found missing on the RM host
Date Thu, 09 Feb 2012 20:54:57 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-3843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Anupam Seth updated MAPREDUCE-3843:
-----------------------------------

    Description: 
This bug was found by Phil Su as part of our testing.

After MAPREDUCE-3354 went in, the Job summary log file seems to have gone missing on the
RM host.

The job summary log appears to be interspersed in yarn-mapredqa-historyserver-<host>.out.

e.g. 
12/02/09 15:57:21 INFO jobhistory.JobSummary:
jobId=job_1328658619341_0011,submitTime=1328802904381,launchTime=1328802909977,firstMapTaskLaunchTime=1328802912116,firstReduceTaskLaunchTime=1328802915074,finishTime=1328802933797,resourc
esPerMap=1024,resourcesPerReduce=2048,numMaps=10,numReduces=10,user=hadoopqa,queue=default,status=KILLED,mapSlotSeconds=0,reduceSlotSeconds=0

1) On the RM with older hadoop version where the job summary log does not exist
mapredqa 10903  0.0  1.2 1424404 210240 ?      Sl   Feb07   0:19 /home/gs/java/jdk64/current/bin/java
-Xmx1000m
-Djava.net.preferIPv4Stack=true
-Djava.library.path=/home/gs/gridre/theoden/share/hadoop/lib/native/Linux-amd64-64:/
home/gs/gridre/theoden/share/hadoop/lib/native/Linux-amd64-64 -Dhadoop.log.dir=/home/gs/var/log/mapredqa
-Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/home/gs/gridre/theoden/share/hadoop -Dhadoop.id.str=mapredqa
-Dhadoop
.root.logger=INFO,console
-Djava.library.path=/home/gs/gridre/theoden/share/hadoop/lib/native/Linux-amd64-64:/home/gs/gridre/theoden/share/hadoop/lib/native/Linux-amd64-64:/home/gs/gridre/theoden/share/hadoop/lib/nat
ive -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dmapred.jobsummary.logger=INFO,console
-Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer

2) On the RM with older hadoop version where the job summary log exists
mapredqa 24851  0.0  0.5 1463280 90516 ?       Sl   Jan25   0:37 /home/gs/java/jdk64/current/bin/java
-Dproc_historyserver -Xmx1000m -Dmapred.jobsummary.logger=INFO,JSA -Dyarn.log.dir=/home/gs/var/log/mapredqa
-Dyarn.log.file=yarn.log -Dyarn.home.dir= -Dyarn.id.str= -Dyarn.root.logger=INFO,console
-Djava.library.path=/home/gs/gridre/shelob/share/hadoop/lib/native/Linux-amd64-64
-Dyarn.policy.file=hadoop-policy.xml -Dyarn.log.dir=/home/gs/var/log/mapredqa
-Dyarn.log.file=yarn-mapredqa-historyserver-<host>.log -Dyarn.home.dir= -Dyarn.id.str=mapredqa
-Dyarn.root.logger=INFO,DRFA -Djava.library.path=/home/gs/gridre/shelob/share/hadoop/lib/native/Linux-amd64-64
-Dyarn.policy.file=hadoop-policy.xml -Dmapred.jobsummary.logger=INFO,JSA -Dhadoop.log.dir=/home/gs/var/log/mapredqa
-Dyarn.log.dir=/home/gs/var/log/mapredqa
-Dhadoop.log.file=yarn-mapredqa-historyserver-<host>.log
-Dyarn.log.file=yarn-mapredqa-historyserver-<host>.log
-Dyarn.home.dir=/home/gs/gridre/shelob/share/hadoop -Dhadoop.root.logger=INFO,DRFA -Dyarn.root.logger=INFO,DRFA
-Djava.library.path=/home/gs/gridre/shelob/share/hadoop/lib/native/Linux-amd64-64 -classpath
/home/gs/gridre/shelob/conf/hadoop:/home/gs/gridre/shelob/conf/hadoop:/home/gs/gridre/shelob/conf/hadoop:/home/gs/gridre/shelob/conf/hadoop:/home/gs/gridre/shelob/share/hadoop/share/hadoop/common/lib/*:/home/gs/gridre/shelob/share/hadoop/share/hadoop/common/*:/home/gs/gridre/shelob/share/hadoop/hadoop-*-capacity-scheduler.jar:/home/gs/gridre/shelob/share/hadoop/hadoop-*-capacity-scheduler.jar:/home/gs/gridre/shelob/share/hadoop/hadoop-*-capacity-scheduler.jar:/home/gs/gridre/shelob/share/hadoop/share/hadoop/hdfs:/home/gs/gridre/shelob/share/hadoop/share/hadoop/hdfs/lib/*:/home/gs/gridre/shelob/share/hadoop/share/hadoop/hdfs/*:/home/gs/gridre/shelob/share/hadoop/share/hadoop/mapreduce/lib/*:/home/gs/gridre/shelob/share/hadoop/share/hadoop/mapreduce/*:/home/gs/java/jdk64/current/lib/tools.jar:/home/gs/gridre/shelob/share/hadoop/share/hadoop/mapreduce/*:/home/gs/gridre/shelob/share/hadoop/share/hadoop/mapreduce/lib/*
org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer

1) On the RM with older hadoop version where the job summary log does not exist
jobhistory ps shows using the option:
-Dmapred.jobsummary.logger=INFO,console 

2) On the RM with older hadoop version where the job summary log exists
jobhistory ps shows using the option:
-Dmapred.jobsummary.logger=INFO,JSA 
-Dmapred.jobsummary.logger=INFO,JSA



  was:JobHistoryServer belongs to mapreduce land.

       Priority: Critical  (was: Blocker)
       Assignee: Anupam Seth  (was: Jonathan Eagles)
        Summary: Job summary log file found missing on the RM host  (was: CLONE - JobHistoryServer
should be started by bin/mapred and not by bin/yarn)
    
> Job summary log file found missing on the RM host
> -------------------------------------------------
>
>                 Key: MAPREDUCE-3843
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3843
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobhistoryserver, mrv2
>    Affects Versions: 0.23.0
>            Reporter: Anupam Seth
>            Assignee: Anupam Seth
>            Priority: Critical
>             Fix For: 0.23.1
>
>
> This bug was found by Phil Su as part of our testing.
> After MAPREDUCE-3354 went in, the Job summary log file seems to have gone missing on
the
> RM host.
> The job summary log appears to be interspersed in yarn-mapredqa-historyserver-<host>.out.

> e.g. 
> 12/02/09 15:57:21 INFO jobhistory.JobSummary:
> jobId=job_1328658619341_0011,submitTime=1328802904381,launchTime=1328802909977,firstMapTaskLaunchTime=1328802912116,firstReduceTaskLaunchTime=1328802915074,finishTime=1328802933797,resourc
> esPerMap=1024,resourcesPerReduce=2048,numMaps=10,numReduces=10,user=hadoopqa,queue=default,status=KILLED,mapSlotSeconds=0,reduceSlotSeconds=0
> 1) On the RM with older hadoop version where the job summary log does not exist
> mapredqa 10903  0.0  1.2 1424404 210240 ?      Sl   Feb07   0:19 /home/gs/java/jdk64/current/bin/java
-Xmx1000m
> -Djava.net.preferIPv4Stack=true
> -Djava.library.path=/home/gs/gridre/theoden/share/hadoop/lib/native/Linux-amd64-64:/
> home/gs/gridre/theoden/share/hadoop/lib/native/Linux-amd64-64 -Dhadoop.log.dir=/home/gs/var/log/mapredqa
> -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/home/gs/gridre/theoden/share/hadoop -Dhadoop.id.str=mapredqa
> -Dhadoop
> .root.logger=INFO,console
> -Djava.library.path=/home/gs/gridre/theoden/share/hadoop/lib/native/Linux-amd64-64:/home/gs/gridre/theoden/share/hadoop/lib/native/Linux-amd64-64:/home/gs/gridre/theoden/share/hadoop/lib/nat
> ive -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dmapred.jobsummary.logger=INFO,console
> -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer
> 2) On the RM with older hadoop version where the job summary log exists
> mapredqa 24851  0.0  0.5 1463280 90516 ?       Sl   Jan25   0:37 /home/gs/java/jdk64/current/bin/java
> -Dproc_historyserver -Xmx1000m -Dmapred.jobsummary.logger=INFO,JSA -Dyarn.log.dir=/home/gs/var/log/mapredqa
> -Dyarn.log.file=yarn.log -Dyarn.home.dir= -Dyarn.id.str= -Dyarn.root.logger=INFO,console
> -Djava.library.path=/home/gs/gridre/shelob/share/hadoop/lib/native/Linux-amd64-64
> -Dyarn.policy.file=hadoop-policy.xml -Dyarn.log.dir=/home/gs/var/log/mapredqa
> -Dyarn.log.file=yarn-mapredqa-historyserver-<host>.log -Dyarn.home.dir= -Dyarn.id.str=mapredqa
> -Dyarn.root.logger=INFO,DRFA -Djava.library.path=/home/gs/gridre/shelob/share/hadoop/lib/native/Linux-amd64-64
> -Dyarn.policy.file=hadoop-policy.xml -Dmapred.jobsummary.logger=INFO,JSA -Dhadoop.log.dir=/home/gs/var/log/mapredqa
> -Dyarn.log.dir=/home/gs/var/log/mapredqa
> -Dhadoop.log.file=yarn-mapredqa-historyserver-<host>.log
> -Dyarn.log.file=yarn-mapredqa-historyserver-<host>.log
> -Dyarn.home.dir=/home/gs/gridre/shelob/share/hadoop -Dhadoop.root.logger=INFO,DRFA -Dyarn.root.logger=INFO,DRFA
> -Djava.library.path=/home/gs/gridre/shelob/share/hadoop/lib/native/Linux-amd64-64 -classpath
> /home/gs/gridre/shelob/conf/hadoop:/home/gs/gridre/shelob/conf/hadoop:/home/gs/gridre/shelob/conf/hadoop:/home/gs/gridre/shelob/conf/hadoop:/home/gs/gridre/shelob/share/hadoop/share/hadoop/common/lib/*:/home/gs/gridre/shelob/share/hadoop/share/hadoop/common/*:/home/gs/gridre/shelob/share/hadoop/hadoop-*-capacity-scheduler.jar:/home/gs/gridre/shelob/share/hadoop/hadoop-*-capacity-scheduler.jar:/home/gs/gridre/shelob/share/hadoop/hadoop-*-capacity-scheduler.jar:/home/gs/gridre/shelob/share/hadoop/share/hadoop/hdfs:/home/gs/gridre/shelob/share/hadoop/share/hadoop/hdfs/lib/*:/home/gs/gridre/shelob/share/hadoop/share/hadoop/hdfs/*:/home/gs/gridre/shelob/share/hadoop/share/hadoop/mapreduce/lib/*:/home/gs/gridre/shelob/share/hadoop/share/hadoop/mapreduce/*:/home/gs/java/jdk64/current/lib/tools.jar:/home/gs/gridre/shelob/share/hadoop/share/hadoop/mapreduce/*:/home/gs/gridre/shelob/share/hadoop/share/hadoop/mapreduce/lib/*
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer
> 1) On the RM with older hadoop version where the job summary log does not exist
> jobhistory ps shows using the option:
> -Dmapred.jobsummary.logger=INFO,console 
> 2) On the RM with older hadoop version where the job summary log exists
> jobhistory ps shows using the option:
> -Dmapred.jobsummary.logger=INFO,JSA 
> -Dmapred.jobsummary.logger=INFO,JSA

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message