hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chu Shih-Tsun <chihchun....@gmail.com>
Subject Re: problem about hbase performance evaluation tool
Date Mon, 22 Sep 2008 02:21:44 GMT

Hi, Ack,

Actually, there were 4 instances-one carried master and three slaves
(regionservers).
I only downloaded one of regionservers' logs to be a sample log. Whatever, I
launched
4 new EC2 instances-1 master and 3 regionservers-with hadoop 0.17.2.1 and
hbase 0.2.1 
as your suggestion. The test was succeeded in writing 16 clients. And I
re-do the test
as the following  steps: disable and then drop the TestTable, use '-rmr' to
wipe the hdfs.
And then the second test was also success with no task-failure.

Thanks for your kindly response and guide!

Chu, Shih-Tsun




stack-3 wrote:
> 
> Thanks for the logs Chu Shih-Tsun.
> 
> Going by the machine names on the logs, two instances were participating 
> in this test.  One carried master, jobtracker, namenode, etc., and the 
> other a datanode, tasktracker, and regionserver.  Is that right?  A 
> single regionserver should be able to sustain 8 concurrent client 
> uploads -- at least it has in my experience though admittedly on 
> different hardware.  If you have 5 instances, I would suggest, one node 
> for jobracker, namenode, and master and then run datanode, TT, and 
> regionserver on the other 4.
> 
> Looking in your master log, all seems fine. Your regionserver log is 
> short.  It only has a few lines in it.   It may not have had DEBUG 
> enabled either (See FAQ for how -- though you have enabled it on the 
> master I can see so you know how to).
> 
> Would suggest wiping your hdfs, and starting over with the new 0.2.1 RC2 
> which you can find here: 
> http://people.apache.org/~stack/hbase-0.2.1-candidate-2/
> 
> Thanks for your patience,
> St.Ack
> 
> 
> 
> Chu Shih-Tsun wrote:
>> Hi Ack,
>>
>> Thanks for your answer.  I tried to launch 5 x-large instances on EC2 and
>> upped the ulimit for the file descriptors. Howerver, the job was still
>> failed while the no. of clients is more than 4 exclude the first test(8
>> clients). The TestTable was disabled and dropped before the next-test.
>> The
>> log files of hadoop/hbase are appended. I would appreciate your helps.
>>
>> http://www.nabble.com/file/p19389689/hbase-test-log.tar.gz
>> hbase-test-log.tar.gz 
>>
>>
>>
>> stack-3 wrote:
>>   
>>> Are you using smal instances up on EC2?  Try x-large instances.
>>>
>>> Dig in on the MR UI.  Find which task failed.  Look at the logs for the
>>> failed task (Make sure DEBUG is enabled).
>>>
>>> Have you have upped your ulimit for file descriptors? See FAQ for how.
>>>
>>> St.Ack
>>>
>>>
>>>
>>> On Wed, Aug 27, 2008 at 11:24 PM, 鞠適存 <chihchun.chu@gmail.com> wrote:
>>>
>>>     
>>>> Hi all,
>>>> I am evaluating the performance and scalability of hbase0.2.0 (work
>>>> with
>>>> hadoop0.17.1) by the PerformanceEvaluation.java in src/test.
>>>> This test is run on AamzonEC2 and I have some question about this tool:
>>>> An
>>>> error would occur when I run more than 4 clients randomWrite
>>>> evaluation.
>>>> The
>>>> error message is shown as follows:
>>>>
>>>>       
>>>
>>>     
>>
>>   
> 
> 
> 

-- 
View this message in context: http://www.nabble.com/problem-about-hbase-performance-evaluation-tool-tp19195200p19600881.html
Sent from the HBase User mailing list archive at Nabble.com.


Mime
View raw message