phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Taylor (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-3788) GLOBAL_MUTATION_BATCH_SIZE should reflect size of chunked batches
Date Fri, 14 Apr 2017 20:57:41 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15969456#comment-15969456
] 

James Taylor edited comment on PHOENIX-3788 at 4/14/17 8:57 PM:
----------------------------------------------------------------

Another, slightly more complex issue is when mutations are processed on the server side. In
this case, the client will still reflect the batch size as all rows across the entire statement,
even though the server will batch it up according to {{phoenix.mutate.batchSize}}. We won't
know definitely on the client what this config parameter is set to on the server. We could
assume it's the same, or we could potentially pass the config value from the client to the
server and base the batching size on that.

An example of such a statement would be:
{code}
connection.setAutoCommit(true);
connection.createStatement().execute("DELETE FROM MY_TABLE WHERE FEED_ID='ABC'");
{code}
Assuming that MY_TABLE has 10K rows with a FEED_ID of 'ABC', though this would be chunked
up into batches of 100 on the server side, the client would still report a mutation size of
10K.


was (Author: jamestaylor):
Another, slightly more complex issue is when mutations are processed on the server side. In
this case, the client will still reflect the batch size as all rows across the entire statement,
even though the server will batch it up according to {{phoenix.mutate.batchSize}}. We won't
know definitely on the client what this config parameter is set to on the server. We could
assume it's the same, or we could potentially pass the config value from the client to the
server and base the batching size on that.

> GLOBAL_MUTATION_BATCH_SIZE should reflect size of chunked batches
> -----------------------------------------------------------------
>
>                 Key: PHOENIX-3788
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3788
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.10.0
>            Reporter: Geoffrey Jacoby
>            Assignee: Geoffrey Jacoby
>
> As part of PHOENIX-541, we started chunking large MutationStates into multiple smaller
batches transparently. However, the relevant metric, GLOBAL_MUTATION_BATCH_SIZE, still is
updated with the total batch size, not the size of each chunk. This means you can't see the
actual batch sizes which are being submitted to HBase. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message