hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Marc Spaggiari (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15320) HBase connector for Kafka Connect
Date Thu, 16 Mar 2017 15:11:42 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15928241#comment-15928241
] 

Jean-Marc Spaggiari commented on HBASE-15320:
---------------------------------------------

The Kafka Connect source pool from the source, and it's not the source pushing to the queue...
That's not what we want, no? We want the source (us) to push to the Kafka queue?


> HBase connector for Kafka Connect
> ---------------------------------
>
>                 Key: HBASE-15320
>                 URL: https://issues.apache.org/jira/browse/HBASE-15320
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Andrew Purtell
>             Fix For: 2.0.0
>
>
> Implement an HBase connector with source and sink tasks for the Connect framework (http://docs.confluent.io/2.0.0/connect/index.html)
available in Kafka 0.9 and later.
> See also: http://www.confluent.io/blog/announcing-kafka-connect-building-large-scale-low-latency-data-pipelines
> An HBase source (http://docs.confluent.io/2.0.0/connect/devguide.html#task-example-source-task)
could be implemented as a replication endpoint or WALObserver, publishing cluster wide change
streams from the WAL to one or more topics, with configurable mapping and partitioning of
table changes to topics.  
> An HBase sink task (http://docs.confluent.io/2.0.0/connect/devguide.html#sink-tasks)
would persist, with optional transformation (JSON? Avro?, map fields to native schema?), Kafka
SinkRecords into HBase tables.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message