hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-4191) Add a testcase for jobhistory
Date Thu, 05 Feb 2009 05:01:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-4191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12670633#action_12670633

Amareshwari Sriramadasu commented on HADOOP-4191:

bq. (1) history file name and location are validated.
This has to done in two locations : mapred.job.hitsory.location and mapred.job.history.user.location

bq. (2) Each line's format is checked against a regex.
I guess this will be done by the parser itself. If anyline doesn't match recType {KEY =" VALUE
"}* pattern, the parser will fail.

bq. (6) We maintain a Map<String, List<String>> of TaskIDs to TaskAttemptIDs and
see if all the atemptIDs are finished before the TaskID finishes.
This will not be the case always. Incase of speculative tasks, TaskAttemptIDs will be killed
after the TaskID finishes.

> Add a testcase for jobhistory
> -----------------------------
>                 Key: HADOOP-4191
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4191
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred, test
>            Reporter: Amar Kamat
>            Assignee: Ravi Gummadi
> Changes in job history might break the history parser which in turn might break some
features like jobtracker-recovery, history-viewer etc. There should be a testcase that catches
these incompatible changes early and informs about the expected change.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message