hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15204) Try to estimate the cell count for adding into WALEdit
Date Sat, 06 Feb 2016 14:22:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15135779#comment-15135779

stack commented on HBASE-15204:

bq. Stack Not very clear on this part. If CP is returning some mutations - we explicitly add
that to a WALEdit. I am just counting if something is there. That is only to get an exact
count because that will also be added to the WALEdit that we create in doMiniBatchMutation.

So, the call to the cp contract should be updated to say something like, "if you add new cells,
you MUST return the count of cells added in the result ... else they will be ignored"... something
like that?

bq.  But if we don't want the cellCountFromCP to be passed then am fine with it too. I thought
since we can use it and get the exact count it is preferable to use it.

It just is odd having this parameter on this macro method. I appreciate the improvement but
having to pass the param down from on high is not clean.

Thanks [~ram_krish]

> Try to estimate the cell count for adding into WALEdit
> ------------------------------------------------------
>                 Key: HBASE-15204
>                 URL: https://issues.apache.org/jira/browse/HBASE-15204
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0, 1.3.0
>         Attachments: HBASE-15204.patch, HBASE-15204_1.patch, HBASE-15204_1.patch, WAlEdit_add_allocation.jpg,
> The write path profiling shows that when we try to add Cells to WALEdits we try to do
a lot of Array copy inorder to grow the Arraylist backing the WALEdits. In a simple one min
profiling of the write path with 50 YCSB threads shows around 261MB of allocation done for
the Array copy to happen. We can try to avoid that. 

This message was sent by Atlassian JIRA

View raw message