Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 72284200B71 for ; Wed, 27 Jul 2016 01:40:22 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 70BBC160AA4; Tue, 26 Jul 2016 23:40:22 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id E4538160AA5 for ; Wed, 27 Jul 2016 01:40:21 +0200 (CEST) Received: (qmail 36119 invoked by uid 500); 26 Jul 2016 23:40:20 -0000 Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list mapreduce-issues@hadoop.apache.org Received: (qmail 36030 invoked by uid 99); 26 Jul 2016 23:40:20 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 26 Jul 2016 23:40:20 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 8B92B2C0D61 for ; Tue, 26 Jul 2016 23:40:20 +0000 (UTC) Date: Tue, 26 Jul 2016 23:40:20 +0000 (UTC) From: "Haibo Chen (JIRA)" To: mapreduce-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (MAPREDUCE-6724) Unsafe conversion from long to int in MergeManagerImpl.unconditionalReserve() MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Tue, 26 Jul 2016 23:40:22 -0000 [ https://issues.apache.org/jira/browse/MAPREDUCE-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Haibo Chen updated MAPREDUCE-6724: ---------------------------------- Attachment: mapreduce6724.007.patch Uploading a patch as [~jira.shegalov] suggested instead of bloating the patch with refactoring. > Unsafe conversion from long to int in MergeManagerImpl.unconditionalReserve() > ----------------------------------------------------------------------------- > > Key: MAPREDUCE-6724 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-6724 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 > Reporter: Haibo Chen > Assignee: Haibo Chen > Attachments: mapreduce6724.001.patch, mapreduce6724.002.patch, mapreduce6724.003.patch, mapreduce6724.004.patch, mapreduce6724.005.patch, mapreduce6724.006.patch, mapreduce6724.007.patch > > > When shuffle is done in memory, MergeManagerImpl converts the requested size to an int to allocate an instance of InMemoryMapOutput. This results in an overflow if the requested size is bigger than Integer.MAX_VALUE and eventually causes the reducer to fail. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org