hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From y_823...@tsmc.com
Subject Re: The occasion to add region server
Date Wed, 24 Mar 2010 05:56:47 GMT
Region Server Count     Taken Time(sec)
20                       1267
15                       1257
10                       1264
9                        1251
8                        1541
7                        1453
6                        1720
5                      no result         2 region server crash


My job is radom read data from Hbase then save it back to hhbase after some
join.
There's not much difference between 9~20 regionservers.
Apparently it can get performance enhance by scaling out regionserver from
5,6  to 9 or above regionservers .




Fleming Chiu(邱宏明)
707-6128
y_823910@tsmc.com
週一無肉日吃素救地球(Meat Free Monday Taiwan)




                                                                                         
                                                            
                      y_823910@tsmc.com                                                  
                                                            
                                               To:      hbase-user@hadoop.apache.org     
                                                            
                      2010/03/24 09:50         cc:      (bcc: Y_823910/TSMC)             
                                                            
                      AM                       Subject: Re: The occasion to add region server
                                                        
                      Please respond to                                                  
                                                            
                      hbase-user                                                         
                                                            
                                                                                         
                                                            
                                                                                         
                                                            




I ran my job while regionservers stop loading regions.
All regionservers almost got the equal amount of regions.
OK, I'll try to shrink my region server to 5 and 3.




Fleming Chiu(邱宏明)
707-6128
y_823910@tsmc.com
週一無肉日吃素救地球(Meat Free Monday Taiwan)





                      Stack

                      <saint.ack@gmail.        To:
"hbase-user@hadoop.apache.org" <hbase-user@hadoop.apache.org>

                      com>                     cc:      (bcc:
Y_823910/TSMC)

                                               Subject: Re: The occasion to
add region server
                      2010/03/24 09:40

                      AM

                      Please respond to

                      hbase-user







You saw no difference. Before starting job all regionservers had equal
amount of regions.  Were regionservers loaded while the job ran.   Try
with 5 regionservers.  If same for sure something is of.



On Mar 23, 2010, at 6:02 PM, y_823910@tsmc.com wrote:

> Hi,
> Our cluster with 20 machines(4 core, 12G Ram, 1U server)
> HBase Version 0.20.2, r834515
> --------------------------------------------
> ZK               :    3
> DataNode         :   20
> Total Region     : 2088
> Region Server    :   10  <<--
> Client Connection:   20
> My job took time : 1264 sec
> --------------------------------------------
> ZK               :    3
> DataNode         :   20
> Total Region     : 2088
> Region Server    :   15  <<--
> Client Connection:   20
> My job took time : 1257 sec
> --------------------------------------------
> ZK               :    3
> DataNode         :   20
> Total Region     : 2088
> Region Server    :   20  <<--
> Client Connection:   20
> My job took time : 1267 sec
>
> According to the above result, it didn't get the performance enhance
> after
> adding region servers.
> why?
> I wonder when is the best occasion to add extra region server.
> In my machine's spec ,maybe a region server will always get a good
> performance with regions under 300.
> Any ideas, thanks.
>
>
>
>
> Fleming Chiu(邱宏明)
> 707-6128
> y_823910@tsmc.com
> 週一無肉日吃素救地球(Meat Free Monday Taiwan)
>
>
> ---
> ---
> ---------------------------------------------------------------------
>                                                         TSMC PROPERTY
> This email communication (and any attachments) is proprietary
> information
> for the sole use of its
> intended recipient. Any unauthorized review, use or distribution by
> anyone
> other than the intended
> recipient is strictly prohibited.  If you are not the intended
> recipient,
> please notify the sender by
> replying to this email, and then delete this email and any copies of
> it
> immediately. Thank you.
> ---
> ---
> ---------------------------------------------------------------------
>
>
>





---------------------------------------------------------------------------

                                                         TSMC PROPERTY

 This email communication (and any attachments) is proprietary information

 for the sole use of its

 intended recipient. Any unauthorized review, use or distribution by anyone

 other than the intended

 recipient is strictly prohibited.  If you are not the intended recipient,

 please notify the sender by

 replying to this email, and then delete this email and any copies of it

 immediately. Thank you.


---------------------------------------------------------------------------








 --------------------------------------------------------------------------- 
                                                         TSMC PROPERTY       
 This email communication (and any attachments) is proprietary information   
 for the sole use of its                                                     
 intended recipient. Any unauthorized review, use or distribution by anyone  
 other than the intended                                                     
 recipient is strictly prohibited.  If you are not the intended recipient,   
 please notify the sender by                                                 
 replying to this email, and then delete this email and any copies of it     
 immediately. Thank you.                                                     
 --------------------------------------------------------------------------- 




Mime
View raw message