hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Corgan <mcor...@hotpads.com>
Subject region doesn't split after 32+ GB
Date Mon, 27 Sep 2010 23:26:12 GMT
I'm sequentially importing ~1 billion small rows (32 byte keys) into a table
called StatAreaModelLink.  I realize that sequential insertion isn't
efficient by design, but I'm not in a hurry so I let it run all weekend.
 It's been proceeding quickly except for ~20s stalls every minute or so.

I also noticed that one regionserver was getting all the load and just
figured that after each split the later region stayed on the current node.
 Turns out the last region stopped splitting altogether and now has a 33gb
store file.

I started importing on 0.20.6, but switched to 0.89.20100726 today.  They
both seem to act similarly.  Using all default settings except VERSIONS=1.

That regionserver's logs constantly say "Compaction requested for region...
because regionserver60020.cacheFlusher"

http://pastebin.com/WJDs7ZbM

Am I doing something wrong, like not giving it enough time to flush/compact?
 There are 23 previous regions that look ok.

The region summary:

StatAreaModelLink
\x00\x00\x07\xD9\x00\x00\x00\x04\x00\x00\x00\x004\x12z\xCF\x00\x00\x00\x09\x00\x00\x00\x00\x00\x00\x00\x00=\xE9C
1285438365987.69034405
stores=1, storefiles=13, storefileSizeMB=34001, memstoreSizeMB=51,
storefileIndexSizeMB=47


Thanks,
Matt

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message