phoenix-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Geoffrey Jacoby (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (PHOENIX-541) Make mutable batch size bytes-based instead of row-based
Date Wed, 30 Nov 2016 00:58:59 GMT

    [ https://issues.apache.org/jira/browse/PHOENIX-541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15707099#comment-15707099
] 

Geoffrey Jacoby edited comment on PHOENIX-541 at 11/30/16 12:58 AM:
--------------------------------------------------------------------

It appears that calculateMutationSize uses the HBase Mutation class's heapSize() calculation,
but the Mutation objects aren't generated until it's time to commit the MutationState, and
I need to estimate the size in bytes before that, because I'd be using it to decide whether
or not it's time to commit in the first place.


was (Author: gjacoby):
It appears that calculateMutationSize uses the HBase Mutation class's heapSize() calculation,
but the Mutation classes aren't generated until it's time to commit the MutationState, and
I need to estimate the size in bytes before that, because I'd be using it to decide whether
or not it's time to commit in the first place.

> Make mutable batch size bytes-based instead of row-based
> --------------------------------------------------------
>
>                 Key: PHOENIX-541
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-541
>             Project: Phoenix
>          Issue Type: Improvement
>    Affects Versions: 3.0-Release
>            Reporter: mujtaba
>            Assignee: Geoffrey Jacoby
>              Labels: newbie
>             Fix For: 4.10.0
>
>
> With current configuration of row-count based mutable batch size, ideal value for batch
size is around 800 rather then current 15k when creating indexes based on memory consumption,
CPU and GC (data size: key: ~60 bytes, 14 integer column in separate CFs)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message