Mailing-List: contact hbase-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of lecompte@gmail.com designates
 209.85.198.235 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:content-transfer-encoding:content-disposition
         :references;
        b=TrE5pIbY2tAJr7bla7KiQQnJzntL92b+qRODzpmgEs5b0AD1tq1Z9gv/jJRZwXBK4X
         fS0tN0wrGetFutzMSPRc1u0mphTlefnGHfm2moPOW/7kOzU3P7Brl8LO19Q4LICEBEZU
         mM9XrhsQwheh5YuPuFmqF26qEv2lFTW3tzN9E=
Message-ID: <6acdaf2a0812191553q259003a2jb58e75e932f17e9e@mail.gmail.com>
Date: Fri, 19 Dec 2008 18:53:06 -0500
From: "Ryan LeCompte" <lecompte@gmail.com>
To: hbase-user@hadoop.apache.org
Subject: Re: Region server memory requirements
In-Reply-To: <6acdaf2a0812191356x5e831dd1r21ab1c68189b7d0e@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <6acdaf2a0812191105g2153e7e7i1900596b38dddb20@mail.gmail.com>
	 <494BF14C.1030202@duboce.net>
	 <6acdaf2a0812191110kade9ea2n4ed0b2bc65a3e8b0@mail.gmail.com>
	 <6acdaf2a0812191306s79546814k6e59322d29e5c4cc@mail.gmail.com>
	 <494C0EC5.4060203@duboce.net> <494C0FAD.8010502@duboce.net>
	 <6acdaf2a0812191356x5e831dd1r21ab1c68189b7d0e@mail.gmail.com>

Hey Stack,

Okay, I was able to get Hadoop 0.19 up and running with Hbase-trunk.
It seems to startup fine, however now when I connect to the hbase
shell and do a simple "list" or try to create a table, I get the
following almost immediately in the hbase master log files:

2008-12-19 18:49:47,408 WARN org.apache.hadoop.ipc.HBaseServer: Out of
Memory in server select
java.lang.OutOfMemoryError: Java heap space
	at org.apache.hadoop.hbase.ipc.HBaseRPC$Invocation.readFields(HBaseRPC.java:142)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Connection.processData(HBaseServer.java:846)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Connection.readAndProcess(HBaseServer.java:813)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.doRead(HBaseServer.java:399)
	at org.apache.hadoop.hbase.ipc.HBaseServer$Listener.run(HBaseServer.java:308)
2008-12-19 18:49:49,888 INFO
org.apache.hadoop.hbase.master.BaseScanner: All 0 .META. region(s)
scanned


Any ideas?

Thanks,
Ryan


On Fri, Dec 19, 2008 at 4:56 PM, Ryan LeCompte <lecompte@gmail.com> wrote:
> Stack,
>
> Thanks for responding. I'm going to experiment with using a simple
> Java object as the cell value as opposed to little tiny string cell
> values and see if that helps. I can't afford to keep increasing the
> memory since I'll eventually run out of space for my other map/reduce
> jobs.
>
> Will keep you all posted.
>
> Thanks,
> Ryan
>
>
> On Fri, Dec 19, 2008 at 4:18 PM, stack <stack@duboce.net> wrote:
>> stack wrote:
>>>
>>> Small cell sizes use up loads of memory.  See HBASE-900 for more on this.
>>>
>>> Other things to try:
>>>
>>> + hbase.io.index.interval is 32 by default.   Set it to 1024 or larger in
>>> your case.  Access may be a little slower -- but maybe not so bad since your
>>> cells are so small -- but the memory-resident indices will be smaller.
>>
>> Oh, pardon me, this is broke in 0.18.x hbase.  See HBASE-981.  Changing the
>> value has no effect.
>> St.Ack
>>
>