hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jan Kunigk (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16415) Replication in different namespace
Date Tue, 27 Jun 2017 11:53:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16064699#comment-16064699
] 

Jan Kunigk commented on HBASE-16415:
------------------------------------

Thanks Guanghao,

I have an implementation working that successfully redirects replication in Pseudo-Distributed
mode, albeit providing a hardcoded redirection configuration (in-code).
Now I am implementing proper configuration for the redirection.
I can think of two options to provide an external configuration for this:

Option 1) is to simply pass the redirections themselves as config keys and have the RedirectingInterClusterReplicationEndpoint
attempt to (also) parse each config key as a ns:tablename spec. 
Map<Sting, String> configuration = new Map<String, String>();
configuration.put("ns1:t1", "ns2:t1");
configuration.put("ns1:tA", "ns1:tB");
This would translate into the following option to the add_peer command from the HBase shell:
CONFIG => { "ns1:t1" => "ns2:t1", "ns1:tA" => "ns1:tB" }

Option 2) is to explicitly name the redirections config items via a name:
Map<Sting, String> configuration = new Map<String, String>();
configuration.put("redirections", "ns1:t1>>ns2:t1,ns1:tA>>ns1:tB");
This would translate into the following option to the add_peer command from the HBase shell:
CONFIG => { "redirections" => "ns1:t1>>ns2:t1,ns1:tA>>ns1:tB"}

Option 1) has the advantage of avoiding special parsing and characters in the configuration
specification.
Option 2) has the advantage of allowing unambiguous keys for the configuration, I am not sure
that "namespacexyz:tableabc" (Option1) is a good choice for a configuration key. It implies
that anything that is not a well-defined key is a redirection rule... Doesn't sound wise,
but I did not want to exclude it...

Opinions? other proposals?

J


> Replication in different namespace
> ----------------------------------
>
>                 Key: HBASE-16415
>                 URL: https://issues.apache.org/jira/browse/HBASE-16415
>             Project: HBase
>          Issue Type: New Feature
>          Components: Replication
>            Reporter: Christian Guegi
>            Assignee: Jan Kunigk
>
> It would be nice to replicate tables from one namespace to another namespace.
> Example:
> Master cluster, namespace=default, table=bar
> Slave cluster, namespace=dr, table=bar
> Replication happens in class ReplicationSink:
>   public void replicateEntries(List<WALEntry> entries, final CellScanner cells,
...){
>     ...
>     TableName table = TableName.valueOf(entry.getKey().getTableName().toByteArray());
>     ...
>     addToHashMultiMap(rowMap, table, clusterIds, m);
>     ...
>     for (Entry<TableName, Map<List<UUID>, List<Row>>> entry :
rowMap.entrySet()) {
>       batch(entry.getKey(), entry.getValue().values());
>     }
>    }



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message