hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stack <st...@duboce.net>
Subject Re: problem about hbase performance evaluation tool
Date Wed, 10 Sep 2008 20:47:54 GMT
Thanks for the logs Chu Shih-Tsun.

Going by the machine names on the logs, two instances were participating 
in this test.  One carried master, jobtracker, namenode, etc., and the 
other a datanode, tasktracker, and regionserver.  Is that right?  A 
single regionserver should be able to sustain 8 concurrent client 
uploads -- at least it has in my experience though admittedly on 
different hardware.  If you have 5 instances, I would suggest, one node 
for jobracker, namenode, and master and then run datanode, TT, and 
regionserver on the other 4.

Looking in your master log, all seems fine. Your regionserver log is 
short.  It only has a few lines in it.   It may not have had DEBUG 
enabled either (See FAQ for how -- though you have enabled it on the 
master I can see so you know how to).

Would suggest wiping your hdfs, and starting over with the new 0.2.1 RC2 
which you can find here: 

Thanks for your patience,

Chu Shih-Tsun wrote:
> Hi Ack,
> Thanks for your answer.  I tried to launch 5 x-large instances on EC2 and
> upped the ulimit for the file descriptors. Howerver, the job was still
> failed while the no. of clients is more than 4 exclude the first test(8
> clients). The TestTable was disabled and dropped before the next-test. The
> log files of hadoop/hbase are appended. I would appreciate your helps.
> http://www.nabble.com/file/p19389689/hbase-test-log.tar.gz
> hbase-test-log.tar.gz 
> stack-3 wrote:
>> Are you using smal instances up on EC2?  Try x-large instances.
>> Dig in on the MR UI.  Find which task failed.  Look at the logs for the
>> failed task (Make sure DEBUG is enabled).
>> Have you have upped your ulimit for file descriptors? See FAQ for how.
>> St.Ack
>> On Wed, Aug 27, 2008 at 11:24 PM, 鞠適存 <chihchun.chu@gmail.com> wrote:
>>> Hi all,
>>> I am evaluating the performance and scalability of hbase0.2.0 (work with
>>> hadoop0.17.1) by the PerformanceEvaluation.java in src/test.
>>> This test is run on AamzonEC2 and I have some question about this tool:
>>> An
>>> error would occur when I run more than 4 clients randomWrite evaluation.
>>> The
>>> error message is shown as follows:

View raw message