Return-Path: X-Original-To: apmail-phoenix-dev-archive@minotaur.apache.org Delivered-To: apmail-phoenix-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9487C18E94 for ; Thu, 4 Feb 2016 02:03:44 +0000 (UTC) Received: (qmail 52416 invoked by uid 500); 4 Feb 2016 02:03:44 -0000 Delivered-To: apmail-phoenix-dev-archive@phoenix.apache.org Received: (qmail 52360 invoked by uid 500); 4 Feb 2016 02:03:44 -0000 Mailing-List: contact dev-help@phoenix.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@phoenix.apache.org Delivered-To: mailing list dev@phoenix.apache.org Received: (qmail 52348 invoked by uid 99); 4 Feb 2016 02:03:44 -0000 Received: from Unknown (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Feb 2016 02:03:44 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 05E9B181B9D for ; Thu, 4 Feb 2016 02:03:44 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -4.449 X-Spam-Level: X-Spam-Status: No, score=-4.449 tagged_above=-999 required=6.31 tests=[KAM_LAZY_DOMAIN_SECURITY=1, RCVD_IN_DNSWL_HI=-5, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, RP_MATCHES_RCVD=-0.429] autolearn=disabled Received: from mx1-eu-west.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id ORPPvN_E9rCP for ; Thu, 4 Feb 2016 02:03:43 +0000 (UTC) Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx1-eu-west.apache.org (ASF Mail Server at mx1-eu-west.apache.org) with SMTP id 5ED4721150 for ; Thu, 4 Feb 2016 02:03:42 +0000 (UTC) Received: (qmail 48844 invoked by uid 99); 4 Feb 2016 02:03:40 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 04 Feb 2016 02:03:40 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 33B9A2C1F6F for ; Thu, 4 Feb 2016 02:03:40 +0000 (UTC) Date: Thu, 4 Feb 2016 02:03:40 +0000 (UTC) From: "James Taylor (JIRA)" To: dev@phoenix.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (PHOENIX-2649) GC/OOM during BulkLoad MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/PHOENIX-2649?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-2649: ---------------------------------- Fix Version/s: 4.7.0 > GC/OOM during BulkLoad > ---------------------- > > Key: PHOENIX-2649 > URL: https://issues.apache.org/jira/browse/PHOENIX-2649 > Project: Phoenix > Issue Type: Bug > Affects Versions: 4.7.0 > Environment: Mac OS, Hadoop 2.7.2, HBase 1.1.2 > Reporter: Sergey Soldatov > Assignee: maghamravikiran > Priority: Critical > Fix For: 4.7.0 > > Attachments: PHOENIX-2649-1.patch, PHOENIX-2649.patch > > > Phoenix fails to complete bulk load of 40Mb csv data with GC heap error during Reduce phase. The problem is in the comparator for TableRowkeyPair. It expects that the serialized value was written using zero-compressed encoding, but at least in my case it was written in regular way. So, trying to obtain length for table name and row key it always get zero and reports that those byte arrays are equal. As the result, the reducer receives all data produced by mappers in one reduce call and fails with OOM. -- This message was sent by Atlassian JIRA (v6.3.4#6332)