hadoop-common-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Hadoop Wiki] Trivial Update of "Hbase/HowToMigrate" by DaveLatham
Date Sat, 17 Oct 2009 23:05:47 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Hadoop Wiki" for change notification.

The "Hbase/HowToMigrate" page has been changed by DaveLatham.
The comment on this change is: minor formatting fix.
http://wiki.apache.org/hadoop/Hbase/HowToMigrate?action=diff&rev1=38&rev2=39

--------------------------------------------------

  ===== Major Compacting all Tables =====
  Before you begin, you MUST run a major compaction on all tables including .META. table.
 A major compaction compacts all store files in a family together dropping deleted and expired
cells.  Major compaction is necessary because the way deletes work changed in 0.20 hbase.
 Migration will not work without your completing major compaction.  Use the shell to start
up major compactions.  For example, the below cluster has only one table named 'a'.  See how
we run a major_compaction on each:
  
+ {{{
- {{{stack@connelly:~/checkouts/hbase/branches/0.19$ ./bin/hbase shell
+ stack@connelly:~/checkouts/hbase/branches/0.19$ ./bin/hbase shell
  HBase Shell; enter 'help<RETURN>' for list of supported commands.
  Version: 0.19.4, r781868, Tue Jul 14 11:27:58 PDT 2009
  hbase(main):001:0> list
@@ -54, +55 @@

  hbase(main):003:0> major_compact '.META.'
  0 row(s) in 0.0245 seconds
  hbase(main):004:0> major_compact '-ROOT-'
- 0 row(s) in 0.0173 seconds}}}
+ 0 row(s) in 0.0173 seconds
+ }}}
  
  In the above, the compaction took no time.  The case will likely be different for you if
you have big tables.
  
  The way to confirm that the major compaction completed is to do a listing of the hbase rootdir
in hdfs.  For each region on the filesystem, each of its stores should have one mapfile only
if major compaction succeeded.  For example, below we list whats under the 'a' table directory
under the hbase rootdir:
  
+ {{{
- {{{/tmp/hbase-stack/hbase/a
+ /tmp/hbase-stack/hbase/a
  /tmp/hbase-stack/hbase/a/1833721875
  /tmp/hbase-stack/hbase/a/1833721875/a
  /tmp/hbase-stack/hbase/a/1833721875/a/info
@@ -71, +74 @@

  /tmp/hbase-stack/hbase/a/1833721875/a/mapfiles/8167759949199600085/data
  /tmp/hbase-stack/hbase/a/1833721875/a/mapfiles/8167759949199600085/.data.crc
  /tmp/hbase-stack/hbase/a/1833721875/a/mapfiles/8167759949199600085/.index.crc
- /tmp/hbase-stack/hbase/a/1833721875/a/mapfiles/8167759949199600085/index}}}
+ /tmp/hbase-stack/hbase/a/1833721875/a/mapfiles/8167759949199600085/index
+ }}}
  
  There is one column family in this table named 'a' (unfortunately, since it muddles the
example, the table name is also 'a').  The table has one region whose encoded name is 1833721875.
 Under this region directory, there are family directories -- in this case there is one for
the 'a' family -- and under each family directory, there is the {{{info}}} -- for store file
metadata -- and the {{{mapfiles}}} directories.  There is only one mapfile in our case above,
named 8167759949199600085 (MapFiles are made of data and index files).
  

Mime
View raw message