hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carlo Curino (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5176) Preemptable annotations (to support preemption in MR)
Date Wed, 29 May 2013 18:52:22 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13669568#comment-13669568
] 

Carlo Curino commented on MAPREDUCE-5176:
-----------------------------------------

I think Chris's proposal to experiment with it in MR and see if it is generalizable is good,
as it helps us to stay concrete since we have a end-to-end story for MR, and it will not be
hard to move this into YARN later if other frameworks will end up needing something similar.
Just for quick reference the prototype code Chris mentioned is in: MAPREDUCE-5189, MAPREDUCE-5192,
MAPREDUCE-5194, MAPREDUCE-5196, MAPREDUCE-5197, and MAPREDUCE-5269. 
                
> Preemptable annotations (to support preemption in MR)
> -----------------------------------------------------
>
>                 Key: MAPREDUCE-5176
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5176
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mrv2
>            Reporter: Carlo Curino
>            Assignee: Carlo Curino
>             Fix For: 2.0.5-beta
>
>         Attachments: MAPREDUCE-5176.1.patch, MAPREDUCE-5176.patch
>
>
> Proposing a patch that introduces a new annotation @Preemptable that represents to the
framework property of user-supplied classes (e.g., Reducer, OutputCommiter). The intended
semantics is that a tagged class is safe to be preempted between invocations. 
> (this is in spirit similar to the Output Contracts of [Nephele/PACT | https://stratosphere.eu/sites/default/files/papers/ComparingMapReduceAndPACTs_11.pdf])

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message