hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Iyappan Srinivasan (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5995) Delete the jobconf copy from the log directory of the JobTracker when the job is retired
Date Wed, 17 Jun 2009 08:24:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12720557#action_12720557
] 

Iyappan Srinivasan commented on HADOOP-5995:
--------------------------------------------

All test scenarios passed.

Test scenarios are described:

Parameters are set as:

 mapred.jobtracker.retirejob.check=10
 mapred.jobtracker.retirejob.interval=10

1) A job is launched and killed. Conf.xml is removed from 
jobtracker/logdir after around 10 seconds. 

2) A job is launched and completed. Conf.xml is removed from 
jobtracker/logdir after around 10 seconds. 

3) After a job is completed and after around 10 seconds, the .xml file will dissapear from
the logs directory and the job details should not be found from the "completed" section of
jobtracker.jsp front page. But it should be found in the job Hisory.

4) After a job is killed and after around 10 seconds, the .xml file will dissapear from the
logs directory and the job details should not be found from the "Failed" section of jobtracker.jsp
front page. But it should be found in the job Hisory.

5) After launching multiple jobs of multiple types ( like sleep , randomwriter), allow them
to complete successfully. After around 10 seconds, the corresponding .xml files will dissapear
from the logs directory and the job details should not be found from the "completed" section
of jobtracker.jsp front page. But they should be found in the job History.

6) After launching multiple jobs of multiple types ( like sleep , randomwriter), and kill
them. After around 10 seconds, the corresponding .xml files will dissapear from the logs directory
and the job details should not be found from the "Failed" section of jobtracker.jsp front
page. But they should be found in the job History.

7) When the job has completed successsfully as well as when it is killed, the Job History
should be showing properly will all links, inlucing job conf link showing proper values.

8) Start a sleep job allow the map to complete 50% and then kill the job tracker. Restart
the job tracker and find the job continuing from where it left off. Check the logs directory.
Ther should be only one .xml file. After the job is complete, it will remove the .xml file
after 10 seconds and that job will be seen in job tracker history link and not in the front
page. Make sure all links are proper including the job conf.

> Delete the jobconf copy from the log directory of the JobTracker when the job is retired
> ----------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5995
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5995
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Devaraj Das
>            Assignee: Amar Kamat
>             Fix For: 0.21.0
>
>         Attachments: HADOOP-5995-v1.0.patch, HADOOP-5995-v1.1.patch
>
>
> The JobTracker (for web-ui viewing purposes), copies the jobconf from the hdfs and store
it in the log directory. The file should be deleted when the job is retired (removed from
memory).

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message