hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amar Kamat (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5394) JobTracker might schedule 2 attempts of the same task with the same attempt id across restarts
Date Wed, 04 Mar 2009 14:51:56 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12678751#action_12678751
] 

Amar Kamat commented on HADOOP-5394:
------------------------------------

bq. This can happen when the jobtracker gets restarted more than once
This can happen when the jobtracker is restarted frequently within a very small time window

bq. This can cause problems not only with the side-effect files but also can cause the jobtracker
to be in an inconsistent state
Actually the job will be in the inconsistent state. The JobTracker should be fine.

Note that this can happen on small cluster where the rate of updates per job is less

> JobTracker might schedule 2 attempts of the same task with the same attempt id across
restarts
> ----------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5394
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5394
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Amar Kamat
>            Assignee: Amar Kamat
>            Priority: Critical
>
> This can happen when the jobtracker gets restarted more than once. In such cases, the
jobtracker depends on the jobhistory file for the next restart count. If the new restart-count
is not flushed to the file then there is a fair chance that upon next restart, the jobtracker
might schedule a new attempt with an existing id. This can cause problems not only with the
side-effect files but also can cause the jobtracker to be in an inconsistent state.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message