hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Enis Soztutar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10504) Define Replication Interface
Date Tue, 13 May 2014 22:25:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10504?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13996979#comment-13996979
] 

Enis Soztutar commented on HBASE-10504:
---------------------------------------

bq. So IIUC you are going a completely different way than what was discussed before, right?
I don't mind it but I want to make sure that we're on the same track.
Indeed. If you want to replicate to SOLR for example, no need to add another proxy layer.
You can have SOLR clients do the transformations, and send the updates directly. Having to
mock a cluster with pluggable sink approach requires to implement some hbase-specific implementation
details (zookeeper interactions, meta, rpc layer etc). 
bq. Might still be nice to offer some tools like removeNonReplicableEdits()
Agreed. I was thinking of doing a Filter interface for that. You can either pass a filter,
or you can implement your own filter and invoke it. I have to dig a bit more for that. 
bq. the basic metrics the handling of a disabled peer, etc, for the other consumers.
In the POC patch, I keep the peer definition, but peers can have pluggable ReplicationConsumer.
You can still enable/disable a peer, get the metrics for the peer coming from the ReplicationSource
for that peer. We should allow the replicationConsumer to put up its own metrics with the
peer name as well.

> Define Replication Interface
> ----------------------------
>
>                 Key: HBASE-10504
>                 URL: https://issues.apache.org/jira/browse/HBASE-10504
>             Project: HBase
>          Issue Type: Task
>            Reporter: stack
>            Assignee: stack
>            Priority: Blocker
>             Fix For: 0.99.0
>
>         Attachments: hbase-10504_wip1.patch
>
>
> HBase has replication.  Fellas have been hijacking the replication apis to do all kinds
of perverse stuff like indexing hbase content (hbase-indexer https://github.com/NGDATA/hbase-indexer)
and our [~toffer] just showed up w/ overrides that replicate via an alternate channel (over
a secure thrift channel between dcs over on HBASE-9360).  This issue is about surfacing these
APIs as public with guarantees to downstreamers similar to those we have on our public client-facing
APIs (and so we don't break them for downstreamers).
> Any input [~phunt] or [~gabriel.reid] or [~toffer]?
> Thanks.
>  



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message