Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Sat, 6 Jun 2015 10:46:01 +0000 (UTC)
From: "Hudson (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <JIRA.12835458.1433436592000.9308.1433587561960@Atlassian.JIRA>
In-Reply-To: <JIRA.12835458.1433436592000@Atlassian.JIRA>
References: <JIRA.12835458.1433436592000@Atlassian.JIRA>
 <JIRA.12835458.1433436592114@arcas>
Subject: [jira] [Commented] (MAPREDUCE-6387) Serialize the recently added
 Task#encryptedSpillKey field at the end
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-6387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14575672#comment-14575672 ] 

Hudson commented on MAPREDUCE-6387:
-----------------------------------

FAILURE: Integrated in Hadoop-Yarn-trunk #950 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/950/])
MAPREDUCE-6387. Serialize the recently added Task#encryptedSpillKey field at the end. (Arun Suresh via kasha) (kasha: rev 6786daab33792c9c426adf0fc6b361f193d356a8)
* hadoop-mapreduce-project/CHANGES.txt
* hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Task.java


> Serialize the recently added Task#encryptedSpillKey field at the end
> --------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6387
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6387
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.7.0
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>            Priority: Minor
>             Fix For: 2.7.1
>
>         Attachments: MAPREDUCE-6387.1.patch
>
>
> There was a recent addition of an {{encryptedSpillKey}} to the Task object. And when serialized, this field was written out somewhere in the middle. This caused deployments that do not use DistributedCache to push job jars before running the job to fail rolling upgrade.
> Although deploying via Distributed Cache is the recommended method, there might still be deployments that use the node local classpath to pick up the MR framework classes (eg. for efficiency purposes, since this does not require the jar being copied to hdfs and then to all the nodes)
> Ensuring that it is the last field written and read when the Task object is serialized would alleviate this issue.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)