hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joep Rottinghuis (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-17018) Spooling BufferedMutator
Date Fri, 04 Nov 2016 02:09:58 GMT
Joep Rottinghuis created HBASE-17018:

             Summary: Spooling BufferedMutator
                 Key: HBASE-17018
                 URL: https://issues.apache.org/jira/browse/HBASE-17018
             Project: HBase
          Issue Type: New Feature
            Reporter: Joep Rottinghuis

For Yarn Timeline Service v2 we use HBase as a backing store.
A big concern we would like to address is what to do if HBase is (temporarily) down, for example
in case of an HBase upgrade.

Most of the high volume writes will be mostly on a best-effort basis, but occasionally we
do a flush. Mainly during application lifecycle events, clients will call a flush on the timeline
service API. In order to handle the volume of writes we use a BufferedMutator. When flush
gets called on our API, we in turn call flush on the BufferedMutator.

We would like our interface to HBase be able to spool the mutations to a filesystems in case
of HBase errors. If we use the Hadoop filesystem interface, this can then be HDFS, gcs, s3,
or any other distributed storage. The mutations can then later be re-played, for example through
a MapReduce job.

This message was sent by Atlassian JIRA

View raw message