hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsuyoshi OZAWA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5269) Preemption of Reducer (and Shuffle) via checkpointing
Date Sat, 14 Feb 2015 15:47:13 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14321539#comment-14321539
] 

Tsuyoshi OZAWA commented on MAPREDUCE-5269:
-------------------------------------------

Hi [~augustorsouza], it happens if we edit different projects at the same time. Jenkins CI
seems to run the tests under directory which includes changed files. Then the tests use with
old jars and you face the problem. For instances, when the tests against hadoop-mapreduce-client-core
run, it will download jars from dependencies. One solution is committing the changes against
IFile before taking this JIRA. Your changes look very large, it's better to split JIRAs into
separate JIRAs from from a viewpoint of reviewers.

FYI: On my local, your test passes too.

> Preemption of Reducer (and Shuffle) via checkpointing
> -----------------------------------------------------
>
>                 Key: MAPREDUCE-5269
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5269
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: mrv2
>            Reporter: Carlo Curino
>            Assignee: Carlo Curino
>         Attachments: MAPREDUCE-5269.2.patch, MAPREDUCE-5269.3.patch, MAPREDUCE-5269.4.patch,
MAPREDUCE-5269.5.patch, MAPREDUCE-5269.6.patch, MAPREDUCE-5269.7.patch, MAPREDUCE-5269.patch
>
>
> This patch tracks the changes in the task runtime (shuffle, reducer context, etc.) that
are required to implement checkpoint-based preemption of reducer tasks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message