Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Date: Fri, 21 Aug 2015 10:22:45 +0000 (UTC)
From: "Tao Jie (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Message-ID: <JIRA.12857811.1440152510000.128939.1440152565955@Atlassian.JIRA>
In-Reply-To: <JIRA.12857811.1440152510000@Atlassian.JIRA>
References: <JIRA.12857811.1440152510000@Atlassian.JIRA>
 <JIRA.12857811.1440152510944@arcas>
Subject: [jira] [Updated] (MAPREDUCE-6459) native task crashes when merging
 spilled file on ppc64
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


     [ https://issues.apache.org/jira/browse/MAPREDUCE-6459?page=3Dcom.atla=
ssian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Tao Jie updated MAPREDUCE-6459:
-------------------------------
    Attachment: ppc64_error.txt

> native task crashes when merging spilled file on ppc64
> ------------------------------------------------------
>
>                 Key: MAPREDUCE-6459
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6459
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.6.0
>         Environment: Linux version 2.6.32-431.el6.ppc64
>            Reporter: Tao Jie
>         Attachments: ppc64_error.txt
>
>
> when running native task on ppc64=EF=BC=8Cmerging spilled files fails sin=
ce we could not deserialize local spill file correctly.
> Function readVLong in WritableUtils.h and Buffers.h, we try to compare a =
char with a number and convert a char to int64_t. It does not work correctl=
y on ppc64 since char definition is different between ppc64 and x86 platfor=
m. On x86 platform char is defined as signed number while on ppc64 char is =
unsigned. As a result, we write EOF marker [-1, -1] at the end of spill par=
tition, but deserialize chars as [255, 255].


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)