Return-Path: Delivered-To: apmail-hadoop-hbase-user-archive@locus.apache.org Received: (qmail 23175 invoked from network); 21 Aug 2008 20:14:29 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 21 Aug 2008 20:14:29 -0000 Received: (qmail 90443 invoked by uid 500); 21 Aug 2008 20:14:26 -0000 Delivered-To: apmail-hadoop-hbase-user-archive@hadoop.apache.org Received: (qmail 90431 invoked by uid 500); 21 Aug 2008 20:14:26 -0000 Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hbase-user@hadoop.apache.org Delivered-To: mailing list hbase-user@hadoop.apache.org Received: (qmail 90420 invoked by uid 99); 21 Aug 2008 20:14:26 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 21 Aug 2008 13:14:26 -0700 X-ASF-Spam-Status: No, hits=-1.0 required=10.0 tests=RCVD_IN_DNSWL_LOW,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of gcjhhu-hbase-user@m.gmane.org designates 80.91.229.2 as permitted sender) Received: from [80.91.229.2] (HELO ciao.gmane.org) (80.91.229.2) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 21 Aug 2008 20:13:29 +0000 Received: from list by ciao.gmane.org with local (Exim 4.43) id 1KWGXd-0004gW-LG for hbase-user@hadoop.apache.org; Thu, 21 Aug 2008 20:13:57 +0000 Received: from adsl-70-142-8-25.dsl.ltrkar.sbcglobal.net ([70.142.8.25]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 21 Aug 2008 20:13:57 +0000 Received: from sales by adsl-70-142-8-25.dsl.ltrkar.sbcglobal.net with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Thu, 21 Aug 2008 20:13:57 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: hbase-user@hadoop.apache.org From: "Billy Pearson" Subject: HBase Block Size should we change it? Date: Thu, 21 Aug 2008 15:13:52 -0500 Lines: 17 Message-ID: Mime-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=original Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: adsl-70-142-8-25.dsl.ltrkar.sbcglobal.net X-MSMail-Priority: Normal X-Newsreader: Microsoft Windows Mail 6.0.6001.18000 X-MimeOLE: Produced By Microsoft MimeOLE V6.0.6001.18049 Sender: news X-Virus-Checked: Checked by ClamAV on apache.org Have we looked at the option to set the default block size of the HStoreFile's? Reading over bigtable they use a default 64KB Block Size and 8KB on tables that do heavy random read's. Should make this an option in Hbase to set the block size? If so should be wet it in the hbase-default.xml or on a table level? In the bigtable paper looks like they set it as a setting on the cluster level sense they have different clusters of server for different data types/apps. I would like to see this looked at as I thank it might help our performance numbers at a cost of using more memory on hadoop namenode for the extra blocks.