hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-1662) Tool to run major compaction on catalog regions when hbase is shutdown
Date Wed, 15 Jul 2009 22:49:14 GMT

     [ https://issues.apache.org/jira/browse/HBASE-1662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-1662:
-------------------------

    Attachment: 1662.patch

Adds some utility to FSUtil -- a DirFilter for FileSystem.getListStatus.

Adds a main to Region.  Here is what usage looks like:

{code}
durruti:cleantrunk stack$ ./bin/hbase org.apache.hadoop.hbase.regionserver.HRegion
Usage: HRegion CATLALOG_TABLE_DIR [major_compact]
Options:
 major_compact  Pass this option to major compact passed region.
Default outputs scan of passed region.
{code}

Here is what happens if you major compact:

{code}
durruti:cleantrunk stack$ ./bin/hbase org.apache.hadoop.hbase.regionserver.HRegion /tmp/hbase-stack/hbase/.META.
major_compact
09/07/15 15:45:58 INFO regionserver.HLog: HLog configuration: blocksize=33554432, rollsize=31876710,
enabled=true, flushlogentries=100, optionallogflushinternal=10000ms
09/07/15 15:45:59 INFO regionserver.HLog: New hlog /tmp/hbase-stack/hlog.META.1247697958961/hlog.dat.1247697958971
09/07/15 15:45:59 DEBUG regionserver.HRegion: Opening region .META.,,1, encoded=1028785192
09/07/15 15:45:59 DEBUG regionserver.Store: loaded /tmp/hbase-stack/hbase/.META./1028785192/historian/4170498238110389438,
isReference=false, sequence id=28, length=987, majorCompaction=true
09/07/15 15:45:59 INFO regionserver.StoreFile: Allocating LruBlockCache with maximum size
199.0m
09/07/15 15:45:59 DEBUG regionserver.Store: loaded /tmp/hbase-stack/hbase/.META./1028785192/info/2332831811545144948,
isReference=false, sequence id=28, length=918, majorCompaction=true
09/07/15 15:45:59 INFO regionserver.HRegion: region .META.,,1/1028785192 available; sequence
id is 29
09/07/15 15:45:59 INFO regionserver.HRegion: Starting major compaction on region .META.,,1
09/07/15 15:45:59 DEBUG regionserver.Store: Started compaction of 1 file(s)  into /tmp/hbase-stack/hbase/.META./compaction.dir/1028785192/5547240455794549466
09/07/15 15:45:59 DEBUG regionserver.Store: Completed major compaction of historian; store
size is 987.0
09/07/15 15:45:59 DEBUG regionserver.Store: Started compaction of 1 file(s)  into /tmp/hbase-stack/hbase/.META./compaction.dir/1028785192/6342464511479146760
09/07/15 15:45:59 DEBUG regionserver.Store: Completed major compaction of info; store size
is 918.0
09/07/15 15:45:59 INFO regionserver.HRegion: compaction completed on region .META.,,1 in 0sec
09/07/15 15:45:59 DEBUG regionserver.HRegion: Closing .META.,,1: compactions & flushes
disabled 
09/07/15 15:45:59 DEBUG regionserver.HRegion: Updates disabled for region, no outstanding
scanners on .META.,,1
09/07/15 15:45:59 DEBUG regionserver.HRegion: No more row locks outstanding on region .META.,,1
09/07/15 15:45:59 DEBUG regionserver.Store: closed historian
09/07/15 15:45:59 DEBUG regionserver.Store: closed info
09/07/15 15:45:59 INFO regionserver.HRegion: Closed .META.,,1
09/07/15 15:45:59 DEBUG regionserver.HLog: closing hlog writer in /tmp/hbase-stack/hlog.META.1247697958961
{code}

> Tool to run major compaction on catalog regions when hbase is shutdown
> ----------------------------------------------------------------------
>
>                 Key: HBASE-1662
>                 URL: https://issues.apache.org/jira/browse/HBASE-1662
>             Project: Hadoop HBase
>          Issue Type: New Feature
>            Reporter: stack
>             Fix For: 0.20.0
>
>         Attachments: 1662.patch
>
>
> We need this tool in 0.19.  Mildly useful in 0.20.
> To migrate (HBASE-1215), all tables need to have had a major compaction done.    This
should work fine for user-space tables as long as no writes coming in.  With catalog tables
its a little different.  They will always have some edits and its tricky getting cluster to
go down with it flushing -- adding more files to region.
> Am adding an HRegion#main that will list catalog region content by default -- may help
debugging -- and then if you add the major_compact option, will major_compact passed region.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message