hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6638) Do not attempt to recover jobs if encrypted spill is enabled
Date Thu, 22 Sep 2016 03:29:20 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15511982#comment-15511982

Karthik Kambatla commented on MAPREDUCE-6638:

Patch looks pretty good. Really like the actionable log messages.

Couple of nits:
# In the following log message, s/recovery/recover:
    if (reducerCount > 0 && spillEncrypted) {
      LOG.info("Not attempting to recovery. Intermediate spill encryption" +
          " is enabled.");
# Consider renaming {{attemptRecoveryIfNotFirstAttempt}} to {{shouldAttemptRecoveryIfNotFirstAttempt}}.
That does become quite long, which makes me think: how about {{shouldAttemptRecovery}}? This
method could also check for the first attempt case and return false. And, the if condition
towards the end of {{processRecovery}} could include a check for first attempt. What do you

> Do not attempt to recover jobs if encrypted spill is enabled
> ------------------------------------------------------------
>                 Key: MAPREDUCE-6638
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6638
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: applicationmaster
>    Affects Versions: 2.7.2
>            Reporter: Karthik Kambatla
>            Assignee: Haibo Chen
>         Attachments: mapreduce6638.001.patch, mapreduce6638.002.patch, mapreduce6638.003.patch,
> Post the fix to CVE-2015-1776, jobs with ecrypted spills enabled cannot be recovered
if the AM fails. We should store the key some place safe so they can actually be recovered.
If there is no "safe" place, at least we should restart the job by re-running all mappers/reducers.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org

View raw message