From hadoop-commits-return-3396-apmail-lucene-hadoop-commits-archive=lucene.apache.org@lucene.apache.org Fri Dec 28 18:42:24 2007 Return-Path: Delivered-To: apmail-lucene-hadoop-commits-archive@locus.apache.org Received: (qmail 98075 invoked from network); 28 Dec 2007 18:42:23 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 28 Dec 2007 18:42:23 -0000 Received: (qmail 43714 invoked by uid 500); 28 Dec 2007 18:42:12 -0000 Delivered-To: apmail-lucene-hadoop-commits-archive@lucene.apache.org Received: (qmail 43683 invoked by uid 500); 28 Dec 2007 18:42:12 -0000 Mailing-List: contact hadoop-commits-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-dev@lucene.apache.org Delivered-To: mailing list hadoop-commits@lucene.apache.org Received: (qmail 43674 invoked by uid 99); 28 Dec 2007 18:42:12 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 28 Dec 2007 10:42:12 -0800 X-ASF-Spam-Status: No, hits=-100.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.130] (HELO eos.apache.org) (140.211.11.130) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 28 Dec 2007 18:41:58 +0000 Received: from eos.apache.org (localhost [127.0.0.1]) by eos.apache.org (Postfix) with ESMTP id DA673D2D6 for ; Fri, 28 Dec 2007 18:42:02 +0000 (GMT) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: Apache Wiki To: hadoop-commits@lucene.apache.org Date: Fri, 28 Dec 2007 18:42:02 -0000 Message-ID: <20071228184202.10406.1691@eos.apache.org> Subject: [Lucene-hadoop Wiki] Trivial Update of "Hbase/FAQ" by stack X-Virus-Checked: Checked by ClamAV on apache.org Dear Wiki user, You have subscribed to a wiki page or wiki category on "Lucene-hadoop Wiki" for change notification. The following page has been changed by stack: http://wiki.apache.org/lucene-hadoop/Hbase/FAQ ------------------------------------------------------------------------------ Running an Hbase loaded w/ more than a few regions, its possible to blow past the environment file handle limit for the user running the process. Running out of file handles is like an OOME, things start to fail in strange ways. To up the users' file handles, edit '''/etc/security/limits.conf''' on all nodes and restart your cluster. - '''6. [[Anchor(6)]] Performance?''' + '''6. [[Anchor(6)]] What can I do to improve hbase performance?''' To improve random-read performance, if you can, try making the hdfs block size smaller (as is suggested in the bigtable paper). By default its 64MB. Try setting it to 8MB. On every random read, hbase has to fetch from hdfs the blocks that contain the wanted row. If your rows are small, much smaller than the hdfs block size, then we'll be fetching a lot of data only to discard the bulk. Meantime the big block fetches and processing consume CPU, network, etc. in the datanodes and hbase client.