hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15204) Try to estimate the cell count for adding into WALEdit
Date Sat, 06 Feb 2016 16:23:39 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15204?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15135860#comment-15135860

ramkrishna.s.vasudevan commented on HBASE-15204:

bq.So, the call to the cp contract should be updated to say something like, "if you add new
cells, you MUST return the count of cells added in the result ... else they will be ignored"...
something like that?
Stack, I think I can explain here. There is nothing am doing with the CP call here. The code
in doPreMutationHook calls all the prehooks and we collect all those WALEdits already. Am
just using that WALEdits and counting the size. There is no need for any contract on the CP
side for doing this. While we collect those WALEdits into the 'batchOp.walEditsFromCoprocessors'
already. If there was no edit added from CP this count is going to be 0. No contract explicitly
added here.
Simple way to not pass this cellcout is just to add those cellCount to the BatchOp directly
and then use a getter to add the actual cellCount. 

> Try to estimate the cell count for adding into WALEdit
> ------------------------------------------------------
>                 Key: HBASE-15204
>                 URL: https://issues.apache.org/jira/browse/HBASE-15204
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver
>            Reporter: ramkrishna.s.vasudevan
>            Assignee: ramkrishna.s.vasudevan
>             Fix For: 2.0.0, 1.3.0
>         Attachments: HBASE-15204.patch, HBASE-15204_1.patch, HBASE-15204_1.patch, WAlEdit_add_allocation.jpg,
> The write path profiling shows that when we try to add Cells to WALEdits we try to do
a lot of Array copy inorder to grow the Arraylist backing the WALEdits. In a simple one min
profiling of the write path with 50 YCSB threads shows around 261MB of allocation done for
the Array copy to happen. We can try to avoid that. 

This message was sent by Atlassian JIRA

View raw message