phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geoffrey Jacoby (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PHOENIX-3788) GLOBAL_MUTATION_BATCH_SIZE should reflect size of chunked batches
Date Mon, 05 Jun 2017 20:39:04 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-3788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16037547#comment-16037547
] 

Geoffrey Jacoby commented on PHOENIX-3788:
------------------------------------------

[~jamestaylor], I tried implementing the approach you explain above, and ran into an issue
on the last step: recording on client-side the chunked bytes the server side used in Delete
scans and UPSERT SELECTs. In both cases the server-side MutationPlan execute returns a MutationState,
but it's empty except for the max batch size and the total mutation count of the operation.
In addition to the total mutation count, I also need the total byte size of all mutations.


However, in MutationPlan.execute I only have access to an iterator of Tuples, and the ColumnProjector,
and neither seems to have a way to estimate the byte size of the underlying keyvalues (since
Tuples don't seem to have an iterator over their KeyValues). 

Is there a simple solution I'm missing, or does this need further enhancement to the Tuple
interface/some other dependency?

> GLOBAL_MUTATION_BATCH_SIZE should reflect size of chunked batches
> -----------------------------------------------------------------
>
>                 Key: PHOENIX-3788
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-3788
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.10.0
>            Reporter: Geoffrey Jacoby
>            Assignee: Geoffrey Jacoby
>             Fix For: 4.11.0
>
>         Attachments: PHOENIX-3788.patch
>
>
> As part of PHOENIX-541, we started chunking large MutationStates into multiple smaller
batches transparently. However, the relevant metric, GLOBAL_MUTATION_BATCH_SIZE, still is
updated with the total batch size, not the size of each chunk. This means you can't see the
actual batch sizes which are being submitted to HBase. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message