hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15320) HBase connector for Kafka Connect
Date Mon, 19 Jun 2017 16:55:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16054320#comment-16054320
] 

Sean Busbey commented on HBASE-15320:
-------------------------------------

This would be a great addition, thanks for working on it!

Please start patches against the master branch. We can figure out what release lines a new
feature can get into once things get hammered out more.

What do we gain by running edits through the proxy servers rather than just putting directly
into the kafka broker directly from the replication system?

We'll need to figure out which pieces we expect downstreamers to customize (maybe new filtering
rules?) and appropriately label things with the InterfaceAudience annotations.

How about some docs?

> HBase connector for Kafka Connect
> ---------------------------------
>
>                 Key: HBASE-15320
>                 URL: https://issues.apache.org/jira/browse/HBASE-15320
>             Project: HBase
>          Issue Type: New Feature
>          Components: Replication
>            Reporter: Andrew Purtell
>            Assignee: Mike Wingert
>              Labels: beginner
>             Fix For: 3.0.0, 2.1.0
>
>         Attachments: HBASE-15320.txt
>
>
> Implement an HBase connector with source and sink tasks for the Connect framework (http://docs.confluent.io/2.0.0/connect/index.html)
available in Kafka 0.9 and later.
> See also: http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines
> An HBase source (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task)
could be implemented as a replication endpoint or WALObserver, publishing cluster wide change
streams from the WAL to one or more topics, with configurable mapping and partitioning of
table changes to topics.  
> An HBase sink task (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks)
would persist, with optional transformation (JSON? Avro?, map fields to native schema?), Kafka
SinkRecords into HBase tables.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message