hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Himanshu Vashishtha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10378) Divide HLog interface into User and Implementor specific interfaces
Date Mon, 20 Jan 2014 22:52:20 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10378?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13876942#comment-13876942

Himanshu Vashishtha commented on HBASE-10378:

Thanks for taking a look at the patch. These are really good questions, and are applicable
for the current WAL implementation as well.

bq. should we have an implemenation for the WALservice where based on the number of hLogs
those many syncer and writer threads need to be started along with the replicaiton services
for them? Currently the HRS just instantiates one HLog and starts them. What do you say?
Yes,  that is definitely one possible implementation.

Re: getWAL() api wrt grouping WALs:
Yes, the grouping logic needs to be extracted and made available, it can be either in HRS,
or in the WAL implementor. One way is to use an array of WALService instances in HRegionServer,
and let the WALGroup return an array to the region server? That way, the region server could
invoke its usual method (append/sync, etc) just like it does currently? The grouping logic
would be present in HRs in this case.
Or, have the grouping logic in WAL impl and rewrite the getWAL to use this grouping knowledge?
i.e., getWAL passes a Hregioninfo to the underlying Group-WAL-impl, and the underlying Group-wal
impl returns a WALService instance based on its grouping. In either case, I see WALGroup can
extend Ă…bstractWAL interface. What do you think? Or, let's chat offline?

> Divide HLog interface into User and Implementor specific interfaces
> -------------------------------------------------------------------
>                 Key: HBASE-10378
>                 URL: https://issues.apache.org/jira/browse/HBASE-10378
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>            Reporter: Himanshu Vashishtha
>         Attachments: 10378-1.patch
> HBASE-5937 introduces the HLog interface as a first step to support multiple WAL implementations.
This interface is a good start, but has some limitations/drawbacks in its current state, such
> 1) There is no clear distinction b/w User and Implementor APIs, and it provides APIs
both for WAL users (append, sync, etc) and also WAL implementors (Reader/Writer interfaces,
etc). There are APIs which are very much implementation specific (getFileNum, etc) and a user
such as a RegionServer shouldn't know about it.
> 2) There are about 14 methods in FSHLog which are not present in HLog interface but are
used at several places in the unit test code. These tests typecast HLog to FSHLog, which makes
it very difficult to test multiple WAL implementations without doing some ugly checks.
> I'd like to propose some changes in HLog interface that would ease the multi WAL story:
> 1) Have two interfaces WAL and WALService. WAL provides APIs for implementors. WALService
provides APIs for users (such as RegionServer).
> 2) A skeleton implementation of the above two interface as the base class for other WAL
implementations (AbstractWAL). It provides required fields for all subclasses (fs, conf, log
dir, etc). Make a minimal set of test only methods and add this set in AbstractWAL.
> 3) HLogFactory returns a WALService reference when creating a WAL instance; if a user
need to access impl specific APIs (there are unit tests which get WAL from a HRegionServer
and then call impl specific APIs), use AbstractWAL type casting,
> 4) Make TestHLog abstract and let all implementors provide their respective test class
which extends TestHLog (TestFSHLog, for example).

This message was sent by Atlassian JIRA

View raw message