hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-533) Region Historian
Date Wed, 28 May 2008 22:50:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-533?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12600636#action_12600636
] 

stack commented on HBASE-533:
-----------------------------

Patch is missing regionserver.jsp so couldn't try it.

In your sample above, why two creations of same region?  Then later, same region is created
again (I'm guessing you copy/pasted this table together just as an illustration?).  I might
not be reading the table correctly.

If compactions, I wonder if we shouldn't have flushes also?  Maybe compactions/flushes should
include how many files compacted and time taken?  Then again, thinking on it, we have an instance
with thousands of regions.  Maybe that'd be too much?    Thousands of regions reporting their
compactions, etc., will put up a heavy load on .META. table.  Maybe we shouldn't report compactions,
just the significant region events by default.    Later we can make the amount of detail recorded
by the historian adjustable.  Next level of detail would include compactions.  Above that
again, it would include flushes.  At finest detail, would record hits per minute, etc.

Here's a few comments on the patch:

+ Move this comment "+  /** The Region Historian task is to keep track of every modification
a region..." to be the class comment for RegionHistorian.
+ Maybe hide the singleton aspect of RegionHistorian inside the class.  Rather than have clients
call 'RegionHistorian.getInstance().addRegionOpen...', instead have them call RegionHistorian.addRegionOpen...
and then inside the method, you'd do your singleton thing.
+ What you need here:

{code}
+      /* TODO
+       * REGION_HISTORIAN_KEYS is used because there is no other for the moment 
+       * to retrieve all version and to have the column key information. To be changed
+       * when HTable.getRow handles versions.
+       */
+      for (byte[] columnKey : REGION_HISTORIAN_KEYS) {
+        Cell[] cells = metaTable.get(Bytes.toBytes(regionName), columnKey,
+            ALL_VERSIONS);
...
{code}

A getRows that would return all members of a column family, all versions?

+ Could event types be done as enums?  E.g. REGION_CREATION, etc.
+ Change these to LOG.info/debug:  System.out.println

Otherwise, this patch looks really great -- very sweet feature.

> Region Historian
> ----------------
>
>                 Key: HBASE-533
>                 URL: https://issues.apache.org/jira/browse/HBASE-533
>             Project: Hadoop HBase
>          Issue Type: Wish
>            Reporter: Bryan Duxbury
>            Assignee: Jean-Daniel Cryans
>            Priority: Minor
>         Attachments: hbase-533-v1.patch
>
>
> Whenever we try to debug region splitting, assignment, compaction, etc. issues, we always
end up having to look in 1-20 different log files for cryptic names of regions and try to
piece together the chain of events ourselves. This is a challenging at best effort most of
the time.
> What would be very useful would be a new utility I've nicknamed the Region Historian.
You give it the text name of a region, and it will track down the log messages relevant to
it in the master and regionserver logs. Then, it will interleave the messages in such a way
that the timestamps correctly list the order of events. The result is a log summary that accurately
describes what happened to a region during it's lifetime, making it much easier to try and
figure out where something went wrong.
> Other things it could do would be replace cryptic log messages with simple events like
"the region was split into a and b", "the region was assigned to server x", and trace the
lineage of a region backwards to its parent before it came into existence.
> I'm sure there are other things we would think up that would be useful as well.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message