hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sreekanth Ramakrishnan (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-964) Inaccurate values in jobSummary logs
Date Thu, 24 Sep 2009 11:25:16 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sreekanth Ramakrishnan updated MAPREDUCE-964:
---------------------------------------------

    Attachment: mapreduce-964-1.patch

Attaching a patch to fix this issue of negative value which is caused by the finish time of
task status not being set during kill. The large values of the seconds was seen because finish
time was set but not the start time. The reason for this was due to the kill signal was sent
to an attempt which was about to be launched but at same time recv a kill signal due to the
job completeion, which results in the path in runner where status of the task is checked and
is found to be killed and we dont launch it but set only the finish time.

The patch fixes the issue by setting finish time only when the start time is set and setting
finish time in kill which was missing in TaskTracker.

Running back to back reliability tests for validating the fix.

> Inaccurate values in jobSummary logs
> ------------------------------------
>
>                 Key: MAPREDUCE-964
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-964
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.20.1
>            Reporter: Rajiv Chittajallu
>            Assignee: Sreekanth Ramakrishnan
>            Priority: Critical
>         Attachments: mapreduce-964-1.patch
>
>
> For some jobs the mapSlotSeconds is incorrect.
> negative value
> 09/09/01 18:31:44 INFOmapred.JobInProgress$JobSummary: jobId=job_200908270718_4568,submitTime=1251823543976,launchTime=1251823554310,finishTime=1251829904565,
           numMaps=7965,numSlotsPerMap=1,numReduces=40,numSlotsPerReduce=1,user=wile,queue=runner,status=SUCCEEDED,
        mapSlotSeconds=-2503133523,reduceSlotsSeconds=186536,clusterMapCapacity=11262,clusterReduceCapacity=3754
> or too high
> 09/09/02 23:59:57 INFO mapred.JobInProgress$JobSummary: jobId=job_200908270718_5861,submitTime=1251935672924,launchTime=1251935687698,finishTime=1251935997949,
           numMaps=1026,numSlotsPerMap=1,numReduces=10,numSlotsPerReduce=1,user=dfsload,queue=gridops,status=SUCCEEDED,
        
> mapSlotSeconds=1251949742,reduceSlotsSeconds=537,clusterMapCapacity=11262,clusterReduceCapacity=3754

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message