hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jan Kunigk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16415) Replication in different namespace
Date Mon, 19 Jun 2017 11:56:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16053870#comment-16053870

Jan Kunigk commented on HBASE-16415:

Thanks Ted.

I am in the process of implementing the logic of entry redirection in RedirectingInterClusterReplicationEndpoint,
but wonder how I should combine that class' replicate implementation with redirection.
I would very much appreciate the community once more, while I see several options:

1. Embed, the logic of redirections into the replicate() method, which would mean
a) Overriding replicate of HBaseInterClusterReplicationEndpoint, but also
b) duplicating the actual code in HBaseInterClusterReplicationEndpoint.replicate(), which
I don't think is a good implementation

2. Implement the logic of redirections in a completely new method redirect(), which needs
to be called from within instances of HBaseInterClusterReplicationEndpoint or RedirectingInterClusterReplicationEndpoint
We could wrap the content of replicate() into a doReplicate() method in HBaseInterClusterReplicationEndpoint.
HBaseInterClusterReplicationEndpoint would then only call doReplicate() in replicate()
RedirectingInterClusterReplicationEndpoint would call redirect(); doReplicate() as part of
However, this solution puts the functionality of redirection way down into HBaseInterClusterReplicationEndpoint,
which may not be a good idea.

3. Implement redirect() as an abstract method in HBaseReplicationEndpoint (which is the abstract
base class for HBaseInterClusterReplicationEndpoint and RegionReplicaReplicationEndpoint.
The redirect() implementation of both HBaseInterClusterReplicationEndpoint and RegionReplicaReplicationEndpoint
would not do anything (optionally logs this). RedirectingInterClusterReplicationEndpoint.replicate()
overrides the empty implementation with actually redirecting entries.
While this would lead to "empty" abstract method implementations, it appears to me as the
cleanest and most straightforward way to achieve the redirection logic to occur in the ReplicationEndpoint.

4. Copy the behaviour of getWALEntryfilter() in BaseReplicationEndpoint (This is more involved,
but avoids empty abstract method implementations):
return filters.isEmpty() ? null : new ChainWALEntryFilter(filters);
    1. Implement a getRedirector() function in BaseReplicationEnpoint, which returns an instance
of a novel Redirector class, implementing the redirections logic.
    2. If no explicit Redirectors are set in the endpoint, return an empty redirector (NullRedirector,
    3. Provide a redirectEntries() function in BaseReplicationEndpoint which will invoke getRedirector()
and Redirector.redirect() _if_ getRedirector() != null
    4. The version in BaseReplicationEnpoint will return an empty redirector (or null) for
getRedirector(), which means that direct subclasses (HBaseInterClusterReplicationEndpoint
or RegionReplicaReplicationEndpoint) will also return null and not redirect anything
    5. The version in RedirectingInterClusterReplicationEndpoint will overwrite getRedirector()
and return a Redirector, which implements 
    6. Embed redirectEntries() into the replicate() method of HBaseInterClusterReplicationEndpoint
(or all other classes who want to reserve the option option of redirection) 

Again, appreciate your feedback, J

> Replication in different namespace
> ----------------------------------
>                 Key: HBASE-16415
>                 URL: https://issues.apache.org/jira/browse/HBASE-16415
>             Project: HBase
>          Issue Type: New Feature
>          Components: Replication
>            Reporter: Christian Guegi
>            Assignee: Jan Kunigk
> It would be nice to replicate tables from one namespace to another namespace.
> Example:
> Master cluster, namespace=default, table=bar
> Slave cluster, namespace=dr, table=bar
> Replication happens in class ReplicationSink:
>   public void replicateEntries(List<WALEntry> entries, final CellScanner cells,
>     ...
>     TableName table = TableName.valueOf(entry.getKey().getTableName().toByteArray());
>     ...
>     addToHashMultiMap(rowMap, table, clusterIds, m);
>     ...
>     for (Entry<TableName, Map<List<UUID>, List<Row>>> entry :
rowMap.entrySet()) {
>       batch(entry.getKey(), entry.getValue().values());
>     }
>    }

This message was sent by Atlassian JIRA

View raw message