hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15204) Try to estimate the cell count for adding into WALEdit
Date Sat, 06 Feb 2016 02:00:42 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15135448#comment-15135448

stack commented on HBASE-15204:

A doPreMutationHook returning a count of cells is unexpected behavior; while convenient, the
return is orthogonal to why we call the CP. We are now dependent on CP implementation to do
work for us.

It is kind of odd that doMiniBatch takes a cell count when the other param is a fat batchOp
with description of what the edit its. If cell count should be anywhere, it should be in batchop?

You don't want to preserve passing replay flag to WALEdit? We need to purge the replay stuff
but in another patch? Not important but it is odd that we retain the WALEdit constructor that
takes a replay flag.

Do we have to add a Cell at a time? Can we not pass the whole list in one go?

I appreciate the savings in garbage but perhaps we can do it in a cleaner way.

> Try to estimate the cell count for adding into WALEdit
> ------------------------------------------------------
>                 Key: HBASE-15204
>                 URL: https://issues.apache.org/jira/browse/HBASE-15204
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0, 1.3.0
>         Attachments: HBASE-15204.patch, HBASE-15204_1.patch, HBASE-15204_1.patch, WAlEdit_add_allocation.jpg,
> The write path profiling shows that when we try to add Cells to WALEdits we try to do
a lot of Array copy inorder to grow the Arraylist backing the WALEdits. In a simple one min
profiling of the write path with 50 YCSB threads shows around 261MB of allocation done for
the Array copy to happen. We can try to avoid that. 

This message was sent by Atlassian JIRA

View raw message