hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rohit Jain <rohit.j...@esgyn.com>
Subject Re: How to stream data out of hbase
Date Wed, 25 Oct 2017 05:00:34 GMT
Besides the fact that it will add overhead to your writes depending on what streaming guarantees
you desire (at-most-once, at-least-once, exactly-once), how many subscribers you want to support,
the restart capabilities you would want in failure scenarios, etc.


On Oct 24, 2017, at 11:44 PM, Josh Elser <elserj@apache.org<mailto:elserj@apache.org>>

The most reliably way (read-as, likely to continue working across HBase releases) would probably
be to implement a custom ReplicationEndpoint.

This would abstract away the logic behind "tail'ing of WALs" and give you some nicer APIs
to leverage. Beware that this would still be a rather significant undertaking that would likely
require you to dig into HBase internals to get correct.

On 10/24/17 4:02 PM, yeshwanth kumar wrote:
i am searching for a way to stream data from hbase,
one way to do is with filters , but i need to query hbase continously,
another way is to read directly from WAL, (i am searching for sample code,
and i found WALReader and WAL.Entry API's.  can i use them directly without
any side effects)
can anyone suggest me a good way to stream data out of hbase, as the write
happens, i want the same data to be pushed to another data source.
please let me know
Can you Imagine what I would do if I could do all I can - Art of War
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message