hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-19486) Automatically flush BufferedMutator after a timeout
Date Mon, 18 Dec 2017 12:12:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-19486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16294895#comment-16294895

Hadoop QA commented on HBASE-19486:

| (x) *{color:red}-1 overall{color}* |
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 11s{color} | {color:blue}
Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  3s{color} | {color:red}
HBASE-19486 does not apply to master. Rebase required? Wrong Branch? See https://yetus.apache.org/documentation/0.6.0/precommit-patchnames
for help. {color} |
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19486 |
| JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12902635/HBASE-19486-20171218-1300.patch
| Console output | https://builds.apache.org/job/PreCommit-HBASE-Build/10528/console |
| Powered by | Apache Yetus 0.6.0   http://yetus.apache.org |

This message was automatically generated.

> Automatically flush BufferedMutator after a timeout 
> ----------------------------------------------------
>                 Key: HBASE-19486
>                 URL: https://issues.apache.org/jira/browse/HBASE-19486
>             Project: HBase
>          Issue Type: Improvement
>          Components: Client
>            Reporter: Niels Basjes
>            Assignee: Niels Basjes
>         Attachments: HBASE-19486-20171212-2117.patch, HBASE-19486-20171218-1229.patch,
> I'm working on several projects where we are doing stream / event type processing instead
of batch type processing. We mostly use Apache Flink and Apache Beam for these projects.
> When we ingest a continuous stream of events and feed that into HBase via a BufferedMutator
this all works fine. The buffer fills up at a predictable rate and we can make sure it flushes
several times per second into HBase by tuning the buffer size.
> We also have situations where the event rate is unpredictable. Some times because the
source is in reality a batch job that puts records into Kafka, sometimes because it is the
"predictable in production" application in our testing environment (where only the dev triggers
a handful of events).
> For these kinds of use cases we need a way to 'force' the BufferedMutator to automatically
flush any records in the buffer even if the buffer is not full.
> I'll put up a pull request with a proposed implementation for review against the master
(i.e. 3.0.0).
> When approved I would like to backport this to the 1.x and 2.x versions of the client
in the same (as close as possible) way.

This message was sent by Atlassian JIRA

View raw message