hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1238) Under upload, region servers are unable to compact when loaded with hundreds of regions
Date Fri, 06 Mar 2009 00:35:56 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12679439#action_12679439
] 

Jean-Daniel Cryans commented on HBASE-1238:
-------------------------------------------

Turns out the patch works well :

{code}
2009-03-05 00:37:54,348 INFO org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced
flushing of web_pages,http://camanomade.wordpress.com/,1236183697332 because global memcache
limit of 676.7m exceeded; currently 646.4m and flushing till 422.9m
2009-03-05 00:38:01,738 INFO org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced
flushing of web_pages,http://astore.amazon.com/blogcarnival23-20/detail/0761104844/178-5338054-2231726,1235269774078
because global memcache limit of 676.7m exceeded; currently 622.2m and flushing till 422.9m
2009-03-05 00:38:02,556 INFO org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced
flushing of web_pages,http://www.fsprostab.com/english/index.htm,1235943861247 because global
memcache limit of 676.7m exceeded; currently 600.9m and flushing till 422.9m
2009-03-05 00:38:48,789 INFO org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced
flushing of web_pages,http://blackbamboohawaii.com/infinity/index.html,1236182065945 because
global memcache limit of 676.7m exceeded; currently 580.9m and flushing till 422.9m
2009-03-05 00:38:49,271 INFO org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced
flushing of web_pages,http://ask.metafilter.com/60882/MidSouth-vacation,1235821461371 because
global memcache limit of 676.7m exceeded; currently 563.1m and flushing till 422.9m
2009-03-05 00:38:49,865 INFO org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced
flushing of web_pages,http://www.galapagosisland.net/travel_insurance.html,1236106016987 because
global memcache limit of 676.7m exceeded; currently 545.2m and flushing till 422.9m
2009-03-05 00:38:50,449 INFO org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced
flushing of web_pages,http://backcountryblog.blogspot.com/2006/10/hucking-cliffs-is-soooo-last-week.html,1236167469547
because global memcache limit of 676.7m exceeded; currently 528.2m and flushing till 422.9m
2009-03-05 00:38:50,985 INFO org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced
flushing of web_pages,http://www.fletcherlake.com/index.htm,1236166974226 because global memcache
limit of 676.7m exceeded; currently 511.3m and flushing till 422.9m
2009-03-05 00:38:51,803 INFO org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced
flushing of web_pages,http://www.eurailblog.com/archives/cat_wales.shtml,1235786159128 because
global memcache limit of 676.7m exceeded; currently 495.4m and flushing till 422.9m
2009-03-05 00:38:52,242 INFO org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced
flushing of web_pages,http://www.island.net/~sointula/accom.html,1236183944276 because global
memcache limit of 676.7m exceeded; currently 479.8m and flushing till 422.9m
2009-03-05 00:38:52,774 INFO org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced
flushing of web_pages,http://www.manukapoint.com/Contact__20Us/__;window.mm_menu_b25ee76db7ed98b94e5945a386e8b470_2.addMenuItem__,1235779612713
because global memcache limit of 676.7m exceeded; currently 465.8m and flushing till 422.9m
2009-03-05 00:38:53,252 INFO org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced
flushing of web_pages,http://wellsbeachme.com/wc/index.htm,1236106056942 because global memcache
limit of 676.7m exceeded; currently 452.4m and flushing till 422.9m
2009-03-05 00:38:55,239 INFO org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced
flushing of web_pages,http://www.greeka.com/saronic/agistri/hotels/agistri-rosys-little-village/index.html,1236104694918
because global memcache limit of 676.7m exceeded; currently 439.3m and flushing till 422.9m
2009-03-05 00:38:55,525 INFO org.apache.hadoop.hbase.regionserver.MemcacheFlusher: Forced
flushing of web_pages,http://www.manukapoint.com/Contact__20Us/__;window.mm_menu_09fa2b8abdc5d7b0d6a4fd2cb2877ef1_2.addMenuItem__,1236219657625
because global memcache limit of 676.7m exceeded; currently 426.4m and flushing till 422.9m
2009-03-05 00:39:04,436 INFO org.apache.hadoop.hbase.regionserver.HRegion: starting  compaction
on region web_pages,http://camanomade.wordpress.com/,1236183697332
2009-03-05 00:39:08,996 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed
on region web_pages,http://camanomade.wordpress.com/,1236183697332 in 4sec
2009-03-05 00:39:08,997 INFO org.apache.hadoop.hbase.regionserver.HRegion: starting  compaction
on region web_pages,http://astore.amazon.com/blogcarnival23-20/detail/0761104844/178-5338054-2231726,1235269774078
2009-03-05 00:39:13,673 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed
on region web_pages,http://astore.amazon.com/blogcarnival23-20/detail/0761104844/178-5338054-2231726,1235269774078
in 4sec
...
{code}

I suggest we commit this for 0.19.1, maybe even for 0.20.0 until a better solution is found.

> Under upload, region servers are unable to compact when loaded with hundreds of regions
> ---------------------------------------------------------------------------------------
>
>                 Key: HBASE-1238
>                 URL: https://issues.apache.org/jira/browse/HBASE-1238
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: regionserver
>    Affects Versions: 0.19.0
>            Reporter: Jean-Daniel Cryans
>            Priority: Blocker
>             Fix For: 0.19.1, 0.20.0
>
>         Attachments: hbase-1238.patch
>
>
> We have a situation where each region server is loaded with 100+ regions, most of them
in the same table. During a long upload of webpages, each memcache gets filled near equally
fast so that the global memcache limit is usually triggered before the max memcache size.
Since that emergency flush does not trigger compactions, the number of store files just keeps
growing until it fails on all kinds of errors.
> We need a better story for this as this is a "normal" situation.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message