hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-533) Region Historian
Date Tue, 03 Jun 2008 00:52:45 GMT

     [ https://issues.apache.org/jira/browse/HBASE-533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

stack updated HBASE-533:

    Attachment: hbase-533-v4.patch

v4 includes changes to migration

There were at least three issues w/ migration as it was:

+ regionhistorian wanted to add to the .META. table.  hbase-660 didn't support .META. edits.
+ The way the regionhistorian migration was written, it was running even when the migration
was in 'check' mode: i.e. readOnly.  It was using metautils to add the new column only the
filesystem hadn't been migrated yet so we weren't finding original region; instead we were
making a new one, one without a meta.  We were blind to the old data.
+ Migrate had a bunch of shared state across migration steps with possibility that migration
done in one step could be overwritten by migration done later

Here are changes I made to v3 to make v4:

  I made the FILE_SYSTEM_VERSION public.

M src/java/org/apache/hadoop/hbase/util/MetaUtils.java
  Made it so pattern is now a MetaUtils instance per migration step rather than one for whole
  Made initialiize private and called by constructor.
  Made all methods call getLog; if doesn't exist, its created.  Makes it so we'll have a log
when needed.
  (getMETARows): Renamed from getMETARowsInROOT.  Now takes a tablename.  Returns root
  region or list of .META. regions.
M src/java/org/apache/hadoop/hbase/util/Migrate.java
  Added documentation to try and give better sense of how this thing is supposed to work --
   different modes, etc., and dangers.
   Removed data members.  Make info local to migration steps so less shared state.
   Moved code out of run into small coherent methods so its easier to see whats going on.
   Added new migration step from v2 to v3.
   Removed the MetaUtils that spanned total migration. Instead let migration steps go get
it as needed.
   Changed the historian migration adding a check if we need to migrate.
M src/java/org/apache/hadoop/hbase/util/FSUtils.java
  (getRootDir): Added.

> Region Historian
> ----------------
>                 Key: HBASE-533
>                 URL: https://issues.apache.org/jira/browse/HBASE-533
>             Project: Hadoop HBase
>          Issue Type: Wish
>            Reporter: Bryan Duxbury
>            Assignee: Jean-Daniel Cryans
>            Priority: Minor
>         Attachments: hbase-533-v1-fix.patch, hbase-533-v1.patch, hbase-533-v2.patch,
hbase-533-v3.patch, hbase-533-v4.patch
> Whenever we try to debug region splitting, assignment, compaction, etc. issues, we always
end up having to look in 1-20 different log files for cryptic names of regions and try to
piece together the chain of events ourselves. This is a challenging at best effort most of
the time.
> What would be very useful would be a new utility I've nicknamed the Region Historian.
You give it the text name of a region, and it will track down the log messages relevant to
it in the master and regionserver logs. Then, it will interleave the messages in such a way
that the timestamps correctly list the order of events. The result is a log summary that accurately
describes what happened to a region during it's lifetime, making it much easier to try and
figure out where something went wrong.
> Other things it could do would be replace cryptic log messages with simple events like
"the region was split into a and b", "the region was assigned to server x", and trace the
lineage of a region backwards to its parent before it came into existence.
> I'm sure there are other things we would think up that would be useful as well.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message