hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jan Kunigk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16415) Replication in different namespace
Date Tue, 20 Jun 2017 09:12:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16055420#comment-16055420
] 

Jan Kunigk commented on HBASE-16415:
------------------------------------

Hi Guanghao,

thanks for the comment. Happy to put it into createBatches() as well, however then notion
of redirection is locked into the RedirectingInterClusterReplicationEndpoint.
May be the behaviour of redirecting entries is functionality, fairly common to a range of
endpoints?

Currently I have actually added a 
{code}
  default int redirectEntries(ReplicateContext context) {
    // This method should be overridden by subclasses which support redirection of source
    // TableNames in the target
...
{code}
to ReplicationEndpoint.java

The invocation would not have to occur in the parent class of, but could be a general "trait"
of the replication logic as part of the shipper thread shortly before replicate is called:
{code}
        // Redirect the edits to another table in the target if the endpoint implements it
and if any
        // redirections are configured for the current batch of edits
        // Returns zero if no implementation is provided
        int redirected = source.getReplicationEndpoint().redirectEntries(replicateContext);
{code}

This could give (maybe future) endpoints other than "InterCluster" type endpoints the ability
to implement the same behaviour in a different context.

I'd be interested about your opinion on the above points.
Again, happy to override createBatches() as well.
Either way, looks like we're very close to finalizing the discussion before I implement a
patch :)

Best, J





> Replication in different namespace
> ----------------------------------
>
>                 Key: HBASE-16415
>                 URL: https://issues.apache.org/jira/browse/HBASE-16415
>             Project: HBase
>          Issue Type: New Feature
>          Components: Replication
>            Reporter: Christian Guegi
>            Assignee: Jan Kunigk
>
> It would be nice to replicate tables from one namespace to another namespace.
> Example:
> Master cluster, namespace=default, table=bar
> Slave cluster, namespace=dr, table=bar
> Replication happens in class ReplicationSink:
>   public void replicateEntries(List<WALEntry> entries, final CellScanner cells,
...){
>     ...
>     TableName table = TableName.valueOf(entry.getKey().getTableName().toByteArray());
>     ...
>     addToHashMultiMap(rowMap, table, clusterIds, m);
>     ...
>     for (Entry<TableName, Map<List<UUID>, List<Row>>> entry :
rowMap.entrySet()) {
>       batch(entry.getKey(), entry.getValue().values());
>     }
>    }



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message