hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-10378) Divide HLog interface into User and Implementor specific interfaces
Date Tue, 04 Nov 2014 18:01:36 GMT

     [ https://issues.apache.org/jira/browse/HBASE-10378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sean Busbey updated HBASE-10378:
--------------------------------
    Release Note: 
HBase internals for the write ahead log have been refactored. Advanced users of HBase should
be aware of the following changes.

Public Audience
  - The Admin API for asking a region server to roll WAL files has changed from a synchronous
command that returns a set of regions the WAL implementation would like flushed into an asynchronous
command that returns nothing. Older clients relying on the former behavior will still be able
to interact with newer servers, but the response body will always contain an empty list of
regions to flush.
  - The shell command "hlog_roll" has been deprecated. Operators should use the "wal_roll"
command instead. This command is subject to the changes described above for the Admin API
to roll WAL files.
  - The command for analyzing write ahead logs has been renamed from 'hlog' to 'wal'. The
old usage is deprecated and will be removed in a future version.
  - Some utility methods in the HBaseTesetingUtility related to testing write-ahead-logs were
changed in incompatible ways. No functionality has been removed, but method names and arguments
have changed. See the HBaseTestingUtility javadoc for details.
  - The HLogInputFormat utility class for processing logs with MapReduce has been deprecated
and will be removed in a future version. Users should switch to the WALInputFormat.
  - The labeling of server metrics on the region server status pages changed. Previously,
the number of backing files for the write ahead log was labeled 'Num. HLog Files'. If you
wish to see this statistic now, please look for the label 'Num. WAL Files.'  If you rely on
JMX for these metrics, their location has not changed.

LimitedPrivate(COPROC) Audience, LimitedPrivate(PHOENIX)
  - The RegionObserver API has been updated. The changes are both binary and source backwards
compatible for coprocessors that use the BaseRegionObserver class. For those that implement
RegionObserver directly the changes are binary backwards compatible. Depending on the internals
of future HBase versions, coprocessors using the deprecated API may not see all WAL related
events. Users are strongly encouraged to update their use of the API; see the RegionObserver
javadoc for details.
  - Classes related to reading WAL entries (ReaderBase, ProtobufLogReader, SequenceFileLogReader)
have changed in a backwards incompatible way. Users who referenced HLog.Reader directly or
HLog.Entry will have to update. These changes do not impact compatibility with extant wal
files.
  - The WALObserver API has been updated. The changes are both binary and source backwards
compatible for coprocessors that use the BaseWALObserver class. For those that implement WALObserver
directly the changes are binary backwards compatible. Depending on the internals of future
HBase versions, coprocessors using the deprecated API may not see all WAL related events.
Users are strongly encouraged to update their use of the API; see the WALObserver javadoc
for details.
 - The WALCoprocessorEnvironment  has changed in a backwards incompatible way. WALObserver
coprocessors that relied on retrieving an object representing the write ahead log instance
will have to be updated.

LimitedPrivate(REPLICATION) Audience
 - The WALEntryFilter API has changed in a backwards incompatible way. Implementers will have
to be updated.
 - The ReplicationEndpoint.ReplicateContext API has changed in a backwards incompatible way.
Implementers who use this interface will have to be updated. These changes do not impact wire
compatibility for replicating between clusters.
 - The HLogKey API is deprecated in favor of the WALKey API. Additionally, the HLogKey API
has changed in a backwards incompatible way by changing from implementing WriteableComparable<HLogKey>
to implementing Writeable and Comparable<WALKey>.

  was:
HBase internals for the write ahead log have been refactored. Advanced users of HBase should
be aware of the following changes.

Public Audience
  - The Admin API for asking a region server to roll WAL files has changed from a synchronous
command that returns a set of regions the WAL implementation would like flushed into an asynchronous
command that returns nothing. Older clients relying on the former behavior will still be able
to interact with newer servers, but the response body will always contain an empty list of
regions to flush.
  - The command for analyzing write ahead logs has been renamed from 'hlog' to 'wal'. The
old usage is deprecated and will be removed in a future version.
  - Some utility methods in the HBaseTesetingUtility related to testing write-ahead-logs were
changed in incompatible ways. No functionality has been removed, but method names and arguments
have changed. See the HBaseTestingUtility javadoc for details.
  - The HLogInputFormat utility class for processing logs with MapReduce has been deprecated
and will be removed in a future version. Users should switch to the WALInputFormat.
  - The labeling of server metrics on the region server status pages changed. Previously,
the number of backing files for the write ahead log was labeled 'Num. HLog Files'. If you
wish to see this statistic now, please look for the label 'Num. WAL Files.'  If you rely on
JMX for these metrics, their location has not changed.

LimitedPrivate(COPROC) Audience, LimitedPrivate(PHOENIX)
  - The RegionObserver API has been updated. The changes are both binary and source backwards
compatible for coprocessors that use the BaseRegionObserver class. For those that implement
RegionObserver directly the changes are binary backwards compatible. Depending on the internals
of future HBase versions, coprocessors using the deprecated API may not see all WAL related
events. Users are strongly encouraged to update their use of the API; see the RegionObserver
javadoc for details.
  - Classes related to reading WAL entries (ReaderBase, ProtobufLogReader, SequenceFileLogReader)
have changed in a backwards incompatible way. Users who referenced HLog.Reader directly or
HLog.Entry will have to update. These changes do not impact compatibility with extant wal
files.
  - The WALObserver API has been updated. The changes are both binary and source backwards
compatible for coprocessors that use the BaseWALObserver class. For those that implement WALObserver
directly the changes are binary backwards compatible. Depending on the internals of future
HBase versions, coprocessors using the deprecated API may not see all WAL related events.
Users are strongly encouraged to update their use of the API; see the WALObserver javadoc
for details.
 - The WALCoprocessorEnvironment  has changed in a backwards incompatible way. WALObserver
coprocessors that relied on retrieving an object representing the write ahead log instance
will have to be updated.

LimitedPrivate(REPLICATION) Audience
 - The WALEntryFilter API has changed in a backwards incompatible way. Implementers will have
to be updated.
 - The ReplicationEndpoint.ReplicateContext API has changed in a backwards incompatible way.
Implementers who use this interface will have to be updated. These changes do not impact wire
compatibility for replicating between clusters.
 - The HLogKey API is deprecated in favor of the WALKey API. Additionally, the HLogKey API
has changed in a backwards incompatible way by changing from implementing WriteableComparable<HLogKey>
to implementing Writeable and Comparable<WALKey>.


> Divide HLog interface into User and Implementor specific interfaces
> -------------------------------------------------------------------
>
>                 Key: HBASE-10378
>                 URL: https://issues.apache.org/jira/browse/HBASE-10378
>             Project: HBase
>          Issue Type: Sub-task
>          Components: wal
>            Reporter: Himanshu Vashishtha
>            Assignee: Sean Busbey
>             Fix For: 2.0.0, 0.99.2
>
>         Attachments: 10378-1.patch, 10378-2.patch
>
>
> HBASE-5937 introduces the HLog interface as a first step to support multiple WAL implementations.
This interface is a good start, but has some limitations/drawbacks in its current state, such
as:
> 1) There is no clear distinction b/w User and Implementor APIs, and it provides APIs
both for WAL users (append, sync, etc) and also WAL implementors (Reader/Writer interfaces,
etc). There are APIs which are very much implementation specific (getFileNum, etc) and a user
such as a RegionServer shouldn't know about it.
> 2) There are about 14 methods in FSHLog which are not present in HLog interface but are
used at several places in the unit test code. These tests typecast HLog to FSHLog, which makes
it very difficult to test multiple WAL implementations without doing some ugly checks.
> I'd like to propose some changes in HLog interface that would ease the multi WAL story:
> 1) Have two interfaces WAL and WALService. WAL provides APIs for implementors. WALService
provides APIs for users (such as RegionServer).
> 2) A skeleton implementation of the above two interface as the base class for other WAL
implementations (AbstractWAL). It provides required fields for all subclasses (fs, conf, log
dir, etc). Make a minimal set of test only methods and add this set in AbstractWAL.
> 3) HLogFactory returns a WALService reference when creating a WAL instance; if a user
need to access impl specific APIs (there are unit tests which get WAL from a HRegionServer
and then call impl specific APIs), use AbstractWAL type casting,
> 4) Make TestHLog abstract and let all implementors provide their respective test class
which extends TestHLog (TestFSHLog, for example).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message