hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michel Segel <michael_se...@hotmail.com>
Subject Re: HTable.put hangs on bulk loading
Date Tue, 26 Apr 2011 15:43:11 GMT
How many regions on a region server?
Table splits? GC tuning parameters?
HBase heap size?

All of these qustions could be important...

Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 26, 2011, at 5:25 AM, Stan Barton <bartx007@gmail.com> wrote:

> 
> I am running into similar problem with HBase 0.90.2. My setting is 6 RSs one
> master server, 3 ZK servers and about 20 clients (on different servers)
> feeding the database with inserts. After few hours and around 2.5 millions
> rows inserted the process simply hangs with no what so ever error indication
> (neither by ZK, master, regionservers nor the clients). All the inserting
> clients stop virtually at the same moment, but HBase it self is not down and
> can be queried. 
> 
> In fact, the clients not even crash on time out exception for its
> connection. I have ran into such problem always when I attempted to run such
> importing progress with the new HBase versions (even 0.90.1). Can anybody
> address this problem? Does other have similar problems?
> 
> I can provide further info about the configuration if needed.
> 
> Stan Barton
> 
> 
> 
> 
> ajay.gov wrote:
>> 
>> Sorry, my server config was not attached. Its here:
>> http://pastebin.com/U41QZGiq
>> 
>> thanks
>> -ajay
>> 
>> 
>> 
>> ajay.gov wrote:
>>> 
>>> I am doing a load test for which I need to load a table with many rows. 
>>> I have a small java program that has a for loop and calls HTable.put.  I
>>> am inserting a map of 2 items into a table that has one column family.
>>> The limit of the for loop is currently 20000. However after 15876 rows
>>> the call to Put hangs. I am using autoFlush on the HTable. Any ideas why
>>> this may happen? 
>>> 
>>> The table configuration:
>>> DESCRIPTION                                          ENABLED                
   
>>> {NAME => 'TABLE2', FAMILIES => [{NAME => 'TABLE2_CF true           
           
>>> 1', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0'                   
        
>>> , COMPRESSION => 'NONE', VERSIONS => '3', TTL => '2                
           
>>> 147483647', BLOCKSIZE => '65536', IN_MEMORY => 'fal                   
        
>>> se', BLOCKCACHE => 'true'}]}
>>> 
>>> The HBase config on the client is the one in the hbase-default.xml. Some
>>> values:
>>> hbase.client.write.buffer=2097152
>>> hbase.client.pause=1000
>>> hbase.client.retries.number=10
>>> 
>>> If i use another client I am able to put items to the table. I am also
>>> able to scan items from the table using the hbase shell.
>>> 
>>> I have attached the server configuratio
>>> I don't see anything in the region server or master logs. I have them
>>> here.
>>> 
>>> The master server log:
>>> 2011-04-06 19:02:40,149 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> RegionManager.rootScanner scanning meta region {server:
>>> 184.106.69.238:60020, regionname: -ROOT-,,0.70236052, startKey: <>}
>>> 2011-04-06 19:02:40,152 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> RegionManager.rootScanner scan of 1 row(s) of meta region {server:
>>> 184.106.69.238:60020, regionname: -ROOT-,,0.70236052, startKey: <>}
>>> complete
>>> 2011-04-06 19:02:40,157 INFO
>>> org.apache.hadoop.hbase.master.ServerManager: 1 region servers, 0 dead,
>>> average load 42.0
>>> 2011-04-06 19:03:15,252 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> RegionManager.metaScanner scanning meta region {server:
>>> 184.106.69.238:60020, regionname: .META.,,1.1028785192, startKey: <>}
>>> 2011-04-06 19:03:15,265 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> RegionManager.metaScanner scan of 40 row(s) of meta region {server:
>>> 184.106.69.238:60020, regionname: .META.,,1.1028785192, startKey: <>}
>>> complete
>>> 2011-04-06 19:03:15,266 INFO org.apache.hadoop.hbase.master.BaseScanner:
>>> All 1 .META. region(s) scanned
>>> 
>>> 
>>> The region server logs:
>>> 2011-04-06 19:02:21,294 DEBUG
>>> org.apache.hadoop.hbase.regionserver.HRegion: Creating region
>>> TABLE2,,1302141740486.010a5ae704ed53f656cbddb8e489162a.
>>> 2011-04-06 19:02:21,295 INFO
>>> org.apache.hadoop.hbase.regionserver.HRegion: Onlined
>>> TABLE2,,1302141740486.010a5ae704ed53f656cbddb8e489162a.; next
>>> sequenceid=1
>>> 
>>> 
>> 
>> 
> 
> -- 
> View this message in context: http://old.nabble.com/HTable.put-hangs-on-bulk-loading-tp31338874p31477194.html
> Sent from the HBase User mailing list archive at Nabble.com.
> 
> 

Mime
View raw message