Return-Path: Delivered-To: apmail-hbase-user-archive@www.apache.org Received: (qmail 66772 invoked from network); 30 Sep 2010 05:23:39 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 30 Sep 2010 05:23:39 -0000 Received: (qmail 63471 invoked by uid 500); 30 Sep 2010 05:23:38 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 63415 invoked by uid 500); 30 Sep 2010 05:23:36 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 63402 invoked by uid 99); 30 Sep 2010 05:23:35 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 30 Sep 2010 05:23:35 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=10.0 tests=RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of alexey@kovyrin.net designates 209.85.214.41 as permitted sender) Received: from [209.85.214.41] (HELO mail-bw0-f41.google.com) (209.85.214.41) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 30 Sep 2010 05:23:30 +0000 Received: by bwz10 with SMTP id 10so1653092bwz.14 for ; Wed, 29 Sep 2010 22:23:08 -0700 (PDT) Received: by 10.204.112.207 with SMTP id x15mr2029774bkp.194.1285824188086; Wed, 29 Sep 2010 22:23:08 -0700 (PDT) MIME-Version: 1.0 Received: by 10.204.99.210 with HTTP; Wed, 29 Sep 2010 22:22:47 -0700 (PDT) X-Originating-IP: [174.119.138.158] In-Reply-To: References: From: Alexey Kovyrin Date: Thu, 30 Sep 2010 01:22:47 -0400 Message-ID: Subject: Re: Can't insert data into hbase, OOME. To: user@hbase.apache.org Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable http://github.com/stumbleupon/hbase/blob/master/src/main/java/org/apache/ha= doop/hbase/regionserver/HRegion.java#L1418 On Thu, Sep 30, 2010 at 1:17 AM, Andrey Stepachev wrote: > Against what version this patch. =C2=A0I can't find "Only 1 KV, does" in > any stumbleupon or > upstream repositories > > > 2010/9/29 Jean-Daniel Cryans : >> The fix is here: http://pastebin.com/zuL23e0U >> >> We're going to do a push to github later today, along with other >> patches that require more testing. >> >> J-D >> >> On Wed, Sep 29, 2010 at 10:54 AM, Andrey Stepachev wr= ote: >>> wow. i'll wait. thanks for reply. >>> >>> 2010/9/29 Jean-Daniel Cryans : >>>> Ok I found the bug, I think it's only in our distro. >>>> >>>> Stay tuned! >>>> >>>> J-D >>>> >>>> On Wed, Sep 29, 2010 at 9:26 AM, Jean-Daniel Cryans wrote: >>>>> Weird indeed, even after the WAL was rolled 4 times (theoretically >>>>> 256MB of data) I don't even see a flush request... although you're >>>>> running at INFO level instead of DEBUG. Could you switch that and sen= d >>>>> us just the full log. >>>>> >>>>> Thanks a lot! >>>>> >>>>> J-D >>>>> >>>>> On Wed, Sep 29, 2010 at 4:25 AM, Andrey Stepachev = wrote: >>>>>> Hi all, >>>>>> >>>>>> I'm stuck. I can't insert any valuable peace of data into hbase. >>>>>> >>>>>> Data is something around ~20mil rows (20G). I try to insert them int= o >>>>>> nondistributed hbase with 4 parallel jobs. >>>>>> MR job run until all memory given to hbase is exhaused and then >>>>>> hbase produces hprof file. As profiler shows, all memory accumulated >>>>>> in MemStore.kvset. >>>>>> I don't understand, why hbase doesn't block untill flush memstore. >>>>>> The same if I give hbase 6Gb or RAM. >>>>>> >>>>>> 6GB gc log http://paste.ubuntu.com/502577/ >>>>>> >>>>>> hadoop: 0.20.2+320 >>>>>> hbase: stumbleupon-20100830 >>>>>> >>>>>> nothing in hbase-site.xml (except hbase.rootdir and zookeeper.quorum= ). >>>>>> >>>>>> >>>>>> >>>>>> Andrey. >>>>>> >>>>> >>>> >>> >> > --=20 Alexey Kovyrin http://kovyrin.net/