hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6638) Do not attempt to recover jobs if encrypted spill is enabled
Date Fri, 23 Sep 2016 18:38:21 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15517217#comment-15517217
] 

Haibo Chen commented on MAPREDUCE-6638:
---------------------------------------

Hi [~hitesh] Thanks very much for your comments! Sorry for the confusing phase in there. I
guess a better way to put it is probably something like 
Avoid recovering progress made in previous attempts if encrypted spill is enabled. 

If recovery and spill encryption are both enabled, the job attempt does not fail. The job
attempt will simply start from scratch because no history file is parsed. If the naming in
the patch is too misleading, I can create a new follow up jira to rename the shouldAttemptRecovery.
How about shouldAttemptToRecoveryProgress?

> Do not attempt to recover jobs if encrypted spill is enabled
> ------------------------------------------------------------
>
>                 Key: MAPREDUCE-6638
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6638
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: applicationmaster
>    Affects Versions: 2.7.2
>            Reporter: Karthik Kambatla
>            Assignee: Haibo Chen
>         Attachments: mapreduce6638.001.patch, mapreduce6638.002.patch, mapreduce6638.003.patch,
mapreduce6638.004.patch, mapreduce6683.005.patch
>
>
> Post the fix to CVE-2015-1776, jobs with ecrypted spills enabled cannot be recovered
if the AM fails. We should store the key some place safe so they can actually be recovered.
If there is no "safe" place, at least we should restart the job by re-running all mappers/reducers.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message