hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-2231) Compaction events should be written to HLog
Date Wed, 24 Apr 2013 04:35:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13640066#comment-13640066

stack commented on HBASE-2231:

bq. .... it seems that IO Fencing for the WAL works

Excellent (You are a better man than me!)

bq.  stack do you remember how did you test and conclude that it does not work? This is pretty
important, so I can dedicate some more time on testing it.

Straight hbase and its bundled hadoop from circa November 2012.

Looking at the differences between v5 and v4, I do not see anything that could have changed
the mechanics unless its the hdfs difference, or very likely, I just wasn't testing properly.

Patch looks great.

+  repeated string compactionInput = 4;
+  repeated string compactionOutput = 5;
+  required string storeHomeDir = 6;

Are these full paths in above or partials?  Hopefully the latter (in case hbase.rootdir gets

This is good change:

-            if (HLogUtil.isMetaFamily(kv.getFamily())) continue;
+            if (WALEdit.isMetaEditFamily(kv.getFamily())) continue;

Do we need this class, CompactionDescriptor?  Why not just use the pb class of same name?
 It has good accessors, etc., already?  Or its constructor should take the pb CompactionDescriptor?

Should toByteArray add pb magic preamble?  (Probably not -- too much).

Is there enough to ensure that this is a compaction edit?

+            if (kv.matchingFamily(WALEdit.METAFAMILY) ||
                   this.getRegionInfo().getEncodedNameAsBytes())) {
+              //this is a special edit, we should handle it
+              CompactionDescriptor compaction = WALEdit.getCompaction(kv);

In other words, any other kinds of edits w/ a METAFAMILY cf?

Is this used?


Patch looks good.

IOFencing works?  I was seeing that I could write the edit anyways even after the master had
recovered the lease.  Maybe it s hbase-7878 returning when it shouldn't have been (we weren't
checking the return flag?)

Thanks Enis.

+1 on commit.

> Compaction events should be written to HLog
> -------------------------------------------
>                 Key: HBASE-2231
>                 URL: https://issues.apache.org/jira/browse/HBASE-2231
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: Todd Lipcon
>            Assignee: stack
>            Priority: Blocker
>              Labels: moved_from_0_20_5
>             Fix For: 0.95.1
>         Attachments: 2231-testcase-0.94.txt, 2231-testcase_v2.txt, 2231-testcase_v3.txt,
2231v2.txt, 2231v3.txt, 2231v4.txt, hbase-2231-testcase.txt, hbase-2231.txt, hbase-2231_v5.patch
> The sequence for a compaction should look like this:
> # Compact region to "new" files
> # Write a "Compacted Region" entry to the HLog
> # Delete "old" files
> This deals with a case where the RS has paused between step 1 and 2 and the regions have
since been reassigned.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message