hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sergey Soldatov (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-18906) Investigate Phoenix usages of Region#waitForXXX APIs
Date Wed, 04 Oct 2017 17:52:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16191702#comment-16191702
] 

Sergey Soldatov edited comment on HBASE-18906 at 10/4/17 5:51 PM:
------------------------------------------------------------------

[~elserj] Sure. Why we need this hack? That's simple. For some cases such as index creation,
deleting  or upserting data basing on the existing data (upsert select queries) we do scan
and batch writes in the coprocessor hook. That lead to very high write load and region server
just unable to handle all those writes and fail with region too busy/not serving exceptions.
Calling flush directly is not the case and makes everything even worst. That terrible code
is supposed to throttle when memstore reached some watermark, so RS will be able to process
flushes in regular way. Requesting flush async and ability to wait for it's completion would
definitely help us a lot. 


was (Author: sergey.soldatov):
[~elserj] Sure. Why we need this hack? That's simple. For some cases such as index creation
or upserting data basing on the existing data (upsert select queries) we do scan and batch
writes in the coprocessor hook. That lead to very high write load and region server just unable
to handle all those writes and fail with region too busy/not serving exceptions. Calling flush
directly is not the case and makes everything even worst. That terrible code is supposed to
throttle when memstore reached some watermark, so RS will be able to process flushes in regular
way. Requesting flush async and ability to wait for it's completion would definitely help
us a lot. 

> Investigate Phoenix usages of Region#waitForXXX APIs
> ----------------------------------------------------
>
>                 Key: HBASE-18906
>                 URL: https://issues.apache.org/jira/browse/HBASE-18906
>             Project: HBase
>          Issue Type: Sub-task
>          Components: Coprocessors
>            Reporter: Anoop Sam John
>            Assignee: Anoop Sam John
>             Fix For: 2.0.0-alpha-4
>
>
> While reviewing HBASE-18183, Andy pointed out that Phoenix uses waitForFlushesAndCompactions
and/or waitForFlushes for diff reasons.  This issue is to see why they need them and whether
alternate ways are possible. This seems to be too much internal stuff and a normal CP hook
calling these would be dangerous.
> If there are alternate ways for Phoenix not to use this and not landing in issues (As
said by Andy) we should suggest/fix for them.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message