hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ajay Govindarajan <agovindara...@yahoo.com>
Subject HTable.put hangs on bulk loading
Date Thu, 07 Apr 2011 05:45:30 GMT

I am doing a load test for which I need to load a table with many rows. 
 I have a small java program that has a for loop and calls HTable.put. 
 I am inserting a map of 2 items into a table that has one column 
family. The limit of the for loop is currently 20000. However after 
15876 rows the call to Put hangs. I am using autoFlush on the HTable. I think its a problem
with Execute threads in the java client.
Any ideas will be greatly appreciated.

The table configuration: 
DESCRIPTION                                          ENABLED      
              
 {NAME => 'TABLE2', FAMILIES => [{NAME => 'TABLE2_CF true                
      
 1', BLOOMFILTER => 'NONE', REPLICATION_SCOPE => '0'                    
        
 , COMPRESSION => 'NONE', VERSIONS => '3', TTL => '2                  
          
 147483647', BLOCKSIZE => '65536', IN_MEMORY => 'fal                    
        
 se', BLOCKCACHE => 'true'}]} 

The HBase config on the client is the one in the hbase-default.xml. Some values: 
hbase.client.write.buffer=2097152 
hbase.client.pause=1000 
hbase.client.retries.number=10 

If i use another client I am able to put items to the table. I am also able to scan items
from the table using the hbase shell.
I don't see anything in the region server or master logs. I have them here.  

The master server log: 
2011-04-06 19:02:40,149 INFO 
org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner 
scanning meta region {server: 184.106.69.238:60020, regionname: 
-ROOT-,,0.70236052, startKey: <>} 
2011-04-06 19:02:40,152 INFO 
org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner 
scan of 1 row(s) of meta region {server: 184.106.69.238:60020, 
regionname: -ROOT-,,0.70236052, startKey: <>} complete 
2011-04-06 19:02:40,157 INFO org.apache.hadoop.hbase.master.ServerManager: 1 region servers,
0 dead, average load 42.0 
2011-04-06 19:03:15,252 INFO 
org.apache.hadoop.hbase.master.BaseScanner: RegionManager.metaScanner 
scanning meta region {server: 184.106.69.238:60020, regionname: 
.META.,,1.1028785192, startKey: <>} 
2011-04-06 19:03:15,265 INFO 
org.apache.hadoop.hbase.master.BaseScanner: RegionManager.metaScanner 
scan of 40 row(s) of meta region {server: 184.106.69.238:60020, 
regionname: .META.,,1.1028785192, startKey: <>} complete 
2011-04-06 19:03:15,266 INFO org.apache.hadoop.hbase.master.BaseScanner: All 1 .META. region(s)
scanned 


The region server logs: 
2011-04-06 19:02:21,294 DEBUG 
org.apache.hadoop.hbase.regionserver.HRegion: Creating region 
TABLE2,,1302141740486.010a5ae704ed53f656cbddb8e489162a. 
2011-04-06 19:02:21,295 INFO 
org.apache.hadoop.hbase.regionserver.HRegion: Onlined 
TABLE2,,1302141740486.010a5ae704ed53f656cbddb8e489162a.; next 
sequenceid=1 
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message