Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Wed, 4 Jun 2014 15:23:02 +0000 (UTC)
From: "Carlo Curino (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <JIRA.12645472.1367344247215.70707.1401895382532@arcas>
In-Reply-To: <JIRA.12645472.1367344247215@arcas>
References: <JIRA.12645472.1367344247215@arcas>
Subject: [jira] [Commented] (MAPREDUCE-5196) CheckpointAMPreemptionPolicy
 implements preemption in MR AM via checkpointing
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-5196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14017761#comment-14017761 ] 

Carlo Curino commented on MAPREDUCE-5196:
-----------------------------------------

Answering Wangda first:

For preemption we had to chop a very large set of changes in patches which are being slowly pushed in (and churn in trunk made some this problematic).

I think your summary is correct. This patch does only propagate the information from the AM to the Tasks. The tasks in turn only log the information. 
The actual checkpointing and release of resources is part of MAPREDUCE-5269. That patch used to work on trunk, but has now some issues, 
and Augusto Souza is looking to fix back into shape, if you have cycles to look at it, help on this is welcome. 

TaskStatus.State.PREEMPTED is set in MAPREDUCE-5269. Again this oddities are by-product of separating a very large chunk of changes in more 
digestible-sized patches.

The idea of this patch is to fix the wiring so that the tasks knows about preemption, but not change the behavior quite yet.


> CheckpointAMPreemptionPolicy implements preemption in MR AM via checkpointing 
> ------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5196
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5196
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mr-am, mrv2
>            Reporter: Carlo Curino
>            Assignee: Carlo Curino
>             Fix For: 3.0.0
>
>         Attachments: MAPREDUCE-5196.1.patch, MAPREDUCE-5196.2.patch, MAPREDUCE-5196.3.patch, MAPREDUCE-5196.patch, MAPREDUCE-5196.patch
>
>
> This JIRA tracks a checkpoint-based AM preemption policy. The policy handles propagation of the preemption requests received from the RM to the appropriate tasks, and bookeeping of checkpoints. Actual checkpointing of the task state is handled in upcoming JIRAs.


--
This message was sent by Atlassian JIRA
(v6.2#6252)