phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geoffrey Jacoby (JIRA)" <>
Subject [jira] [Commented] (PHOENIX-3788) GLOBAL_MUTATION_BATCH_SIZE should reflect size of chunked batches
Date Mon, 05 Jun 2017 20:39:04 GMT


Geoffrey Jacoby commented on PHOENIX-3788:

[~jamestaylor], I tried implementing the approach you explain above, and ran into an issue
on the last step: recording on client-side the chunked bytes the server side used in Delete
scans and UPSERT SELECTs. In both cases the server-side MutationPlan execute returns a MutationState,
but it's empty except for the max batch size and the total mutation count of the operation.
In addition to the total mutation count, I also need the total byte size of all mutations.

However, in MutationPlan.execute I only have access to an iterator of Tuples, and the ColumnProjector,
and neither seems to have a way to estimate the byte size of the underlying keyvalues (since
Tuples don't seem to have an iterator over their KeyValues). 

Is there a simple solution I'm missing, or does this need further enhancement to the Tuple
interface/some other dependency?

> GLOBAL_MUTATION_BATCH_SIZE should reflect size of chunked batches
> -----------------------------------------------------------------
>                 Key: PHOENIX-3788
>                 URL:
>             Project: Phoenix
>          Issue Type: Bug
>    Affects Versions: 4.10.0
>            Reporter: Geoffrey Jacoby
>            Assignee: Geoffrey Jacoby
>             Fix For: 4.11.0
>         Attachments: PHOENIX-3788.patch
> As part of PHOENIX-541, we started chunking large MutationStates into multiple smaller
batches transparently. However, the relevant metric, GLOBAL_MUTATION_BATCH_SIZE, still is
updated with the total batch size, not the size of each chunk. This means you can't see the
actual batch sizes which are being submitted to HBase. 

This message was sent by Atlassian JIRA

View raw message