hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Flavio Junqueira (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5937) Refactor HLog into an interface.
Date Mon, 01 Oct 2012 15:51:08 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13466886#comment-13466886

Flavio Junqueira commented on HBASE-5937:

Thanks a lot for the review, Stack, and Ted for the additions. Here are some more comments:

bq. How is HLogSplitter going to work w/ different implementations? Currently you punt and
refer to FSHLog?

The splitter might not be necessary for every implementation, so I don't think it is a good
idea to make it part of HLog. Since BookKeeper is optimized for writing multiple parallel
streams efficiently, my perspective is that we want to benefit from this feature to avoid
having to split upon recovery. This is my understanding at least, and it would be great to
have a clarification if it is incorrect.

If we assume that a splitter is necessary for every filesystem-based wal, then we could have
another interface extending HLog and defining methods related to splitting. If not, then the
splitter could specific to FSHLog. How does it sound? 

bq. Yeah, whats this crazy thing supposed to do? resetLogReaderClass

I can't remember now where I got it from, but it is used in TestHLogSplit only to reset the
class we use for the HLog reader. We have the ability to configure the class we use for the
reader and the writer.

bq. FYI, next time, you could use Bytes.toBytes in HBase under util to do this: "HBASE::CACHEFLUSH".getBytes(HConstants.UTF8_ENCODING);

Ok, thanks for the hint. I don't think I have added that statement, but I'm happy to change

The following from HLogUtil I'd think belong in an implementation:
getReader // Why this in util and in factory too? .... or, its commented out along w/ createWriter?
getRegionDirRecoveredEditsDir // Can do this later... seems like I ask the implementation
for reader on the edits... 
getSplitEditFilesSorted // Yeah, this would be in the HLog splitting implementation... or
not exposed

Are you suggesting that we make them static methods of say FSHLog instead of having them in
HLogUtil? They seem to be specific to the current trunk implementation of HLog, so it seems
right to do it. The idea behind HLogUtil was to have all static methods in HLog originally.
I can either move them now or create a jira to revisit HLogUtil. 

So, what you want to do? If tests pass and you are fairly sure you have not done anything
to change our wal write speed – it doesn't look like you have changed any of that, then
if you want to do a part 1 commit and then open new issue to address outstanding, improving
WAL Interface, etc., I'd be +1 on that.

I don't think we have changed anything that would affect your wal speed, at least not consciously.
The follow-up task I believe we need to work on, and they could be jiras directly, are:

# Rename HLog interface to WAL as you suggested;
# Revisit methods of the HLog interface;
# Revisit methods of HLogUtil and HLogMetrics;
# Revisit HLogSplitter.
# Implement BKHLog.

> Refactor HLog into an interface.
> --------------------------------
>                 Key: HBASE-5937
>                 URL: https://issues.apache.org/jira/browse/HBASE-5937
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Li Pi
>            Assignee: Flavio Junqueira
>            Priority: Minor
>         Attachments: 5937-hlog-with-javadoc.txt, HBASE-5937.patch, HBASE-5937.patch,
HBASE-5937.patch, HBASE-5937.patch, HBASE-5937.patch, org.apache.hadoop.hbase.client.TestMultiParallel-output.txt
> What the summary says. Create HLog interface. Make current implementation use it.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message