hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stan Barton <bartx...@gmail.com>
Subject Re: HTable.put hangs on bulk loading
Date Tue, 26 Apr 2011 10:25:27 GMT

I am running into similar problem with HBase 0.90.2. My setting is 6 RSs one
master server, 3 ZK servers and about 20 clients (on different servers)
feeding the database with inserts. After few hours and around 2.5 millions
rows inserted the process simply hangs with no what so ever error indication
(neither by ZK, master, regionservers nor the clients). All the inserting
clients stop virtually at the same moment, but HBase it self is not down and
can be queried. 

In fact, the clients not even crash on time out exception for its
connection. I have ran into such problem always when I attempted to run such
importing progress with the new HBase versions (even 0.90.1). Can anybody
address this problem? Does other have similar problems?

I can provide further info about the configuration if needed.

Stan Barton




ajay.gov wrote:
> 
> Sorry, my server config was not attached. Its here:
> http://pastebin.com/U41QZGiq
> 
> thanks
> -ajay
> 
> 
> 
> ajay.gov wrote:
>> 
>> I am doing a load test for which I need to load a table with many rows. 
>> I have a small java program that has a for loop and calls HTable.put.  I
>> am inserting a map of 2 items into a table that has one column family.
>> The limit of the for loop is currently 20000. However after 15876 rows
>> the call to Put hangs. I am using autoFlush on the HTable. Any ideas why
>> this may happen? 
>> 
>> The table configuration:
>> DESCRIPTION                                          ENABLED                    
>>  {NAME => 'TABLE2', FAMILIES => [{NAME => 'TABLE2_CF true              
        
>>  1', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0'                      
     
>>  , COMPRESSION => 'NONE', VERSIONS => '3', TTL => '2                   
        
>>  147483647', BLOCKSIZE => '65536', IN_MEMORY => 'fal                      
     
>>  se', BLOCKCACHE => 'true'}]}
>> 
>> The HBase config on the client is the one in the hbase-default.xml. Some
>> values:
>> hbase.client.write.buffer=2097152
>> hbase.client.pause=1000
>> hbase.client.retries.number=10
>> 
>> If i use another client I am able to put items to the table. I am also
>> able to scan items from the table using the hbase shell.
>> 
>> I have attached the server configuratio
>> I don't see anything in the region server or master logs. I have them
>> here.
>> 
>> The master server log:
>> 2011-04-06 19:02:40,149 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> RegionManager.rootScanner scanning meta region {server:
>> 184.106.69.238:60020, regionname: -ROOT-,,0.70236052, startKey: <>}
>> 2011-04-06 19:02:40,152 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> RegionManager.rootScanner scan of 1 row(s) of meta region {server:
>> 184.106.69.238:60020, regionname: -ROOT-,,0.70236052, startKey: <>}
>> complete
>> 2011-04-06 19:02:40,157 INFO
>> org.apache.hadoop.hbase.master.ServerManager: 1 region servers, 0 dead,
>> average load 42.0
>> 2011-04-06 19:03:15,252 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> RegionManager.metaScanner scanning meta region {server:
>> 184.106.69.238:60020, regionname: .META.,,1.1028785192, startKey: <>}
>> 2011-04-06 19:03:15,265 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> RegionManager.metaScanner scan of 40 row(s) of meta region {server:
>> 184.106.69.238:60020, regionname: .META.,,1.1028785192, startKey: <>}
>> complete
>> 2011-04-06 19:03:15,266 INFO org.apache.hadoop.hbase.master.BaseScanner:
>> All 1 .META. region(s) scanned
>> 
>> 
>> The region server logs:
>> 2011-04-06 19:02:21,294 DEBUG
>> org.apache.hadoop.hbase.regionserver.HRegion: Creating region
>> TABLE2,,1302141740486.010a5ae704ed53f656cbddb8e489162a.
>> 2011-04-06 19:02:21,295 INFO
>> org.apache.hadoop.hbase.regionserver.HRegion: Onlined
>> TABLE2,,1302141740486.010a5ae704ed53f656cbddb8e489162a.; next
>> sequenceid=1
>> 
>> 
> 
> 

-- 
View this message in context: http://old.nabble.com/HTable.put-hangs-on-bulk-loading-tp31338874p31477194.html
Sent from the HBase User mailing list archive at Nabble.com.


Mime
View raw message