Return-Path: Delivered-To: apmail-hadoop-hbase-user-archive@minotaur.apache.org Received: (qmail 62213 invoked from network); 15 May 2009 20:11:59 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 15 May 2009 20:11:59 -0000 Received: (qmail 21653 invoked by uid 500); 15 May 2009 20:11:59 -0000 Delivered-To: apmail-hadoop-hbase-user-archive@hadoop.apache.org Received: (qmail 21596 invoked by uid 500); 15 May 2009 20:11:59 -0000 Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hbase-user@hadoop.apache.org Delivered-To: mailing list hbase-user@hadoop.apache.org Received: (qmail 21586 invoked by uid 99); 15 May 2009 20:11:58 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 May 2009 20:11:58 +0000 X-ASF-Spam-Status: No, hits=1.1 required=10.0 tests=FORGED_HOTMAIL_RCVD2,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of lists@nabble.com designates 216.139.236.158 as permitted sender) Received: from [216.139.236.158] (HELO kuber.nabble.com) (216.139.236.158) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 May 2009 20:11:49 +0000 Received: from isper.nabble.com ([192.168.236.156]) by kuber.nabble.com with esmtp (Exim 4.63) (envelope-from ) id 1M53kf-00074k-2p for hbase-user@hadoop.apache.org; Fri, 15 May 2009 13:11:29 -0700 Message-ID: <23566568.post@talk.nabble.com> Date: Fri, 15 May 2009 13:11:29 -0700 (PDT) From: llpind To: hbase-user@hadoop.apache.org Subject: Loading large resultset into HBase MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Nabble-From: sonny_heer@hotmail.com X-Virus-Checked: Checked by ClamAV on apache.org Hey all, I'm loading data from a DB into HBase. I have a single java process iterating over a ResultSet. After about 10,000 rows i do a BatchUpdate. I've changed the Heap size of both Hadoop & HBase to 2000. Setup: 0.19.1. 1 box with master and secondary. 3 boxes with HRegionServer. Problem 1: The load seems to be unblananced: Address Start Code Load 1:60020 1242415770566 requests=0, regions=1, usedHeap=39, maxHeap=1777 2:60020 1242415770417 requests=3, regions=3, usedHeap=52, maxHeap=1777 3:60020 1242415770273 requests=1, regions=3, usedHeap=604, maxHeap=1777 Total: servers: 3 requests=4, regions=7 Problem 2: Around 10 Million rows, the upload starts to slow down. The upload is still going, so i'll update on what happens. -- View this message in context: http://www.nabble.com/Loading-large-resultset-into-HBase-tp23566568p23566568.html Sent from the HBase User mailing list archive at Nabble.com.