hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Spiegelberg (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3433) Remove the KV copy of every KV in Scan; introduced by HBASE-3232 (why doesn't keyonlyfilter make copies rather than mutate -- HBASE-3211)?
Date Tue, 11 Jan 2011 22:37:47 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12980392#action_12980392

Nicolas Spiegelberg commented on HBASE-3433:

dj_ryan: nspiegelberg: extra copy of all the queried data does make a big difference
[12:04pm] dj_ryan: now im goign to have to spend an hour or two profiling to find if it is
the case or not
[12:04pm] nspiegelberg: an extra shallow copy?
[12:04pm] nspiegelberg: I agree that an extra deep copy does
[12:04pm] dj_ryan: are you sure that's a shallow copy?
[12:04pm] nspiegelberg: yes
[12:04pm] dj_ryan: hmm
[12:04pm] dj_ryan: that might have minimal impact
[12:05pm] dj_ryan: i thought it was a deep copy actually.
[12:05pm] dj_ryan: but you'd be surprised at what can affect performance.
[12:05pm] nspiegelberg: that brings up my next point...
[12:05pm] nspiegelberg: I want to add a softCopy() function to KeyValue 
[12:05pm] dj_ryan: which does what
[12:05pm] dj_ryan: and why woudl we use it
[12:06pm] nspiegelberg: because dj_ryan doesn't know when we do a shallow copy.  I oftentimes
don't either.  it's just easier to look at the function and know for sure what it does
[12:06pm] dj_ryan: hmm
[12:06pm] dj_ryan: ha
[12:06pm] dj_ryan: it would make sense to have a kv.copy
[12:06pm] dj_ryan: rather than what the code is doing
[12:06pm] dj_ryan: what with the this and that
[12:07pm] nspiegelberg: but kv.copy should do a deepCopy, correct?
[12:07pm] nspiegelberg: I don't mind 2 deepCopy() and softCopy() functions either
[12:08pm] dj_ryan: well yes
[12:08pm] dj_ryan: i guess deep/soft
[12:08pm] dj_ryan: or whatever might be a better nomenclature
[12:08pm] nspiegelberg: explicit is best.  I chose deep and soft because I've written too
much python and both words are short

> Remove the KV copy of every KV in Scan; introduced by HBASE-3232 (why doesn't keyonlyfilter
make copies rather than mutate -- HBASE-3211)?
> ------------------------------------------------------------------------------------------------------------------------------------------
>                 Key: HBASE-3433
>                 URL: https://issues.apache.org/jira/browse/HBASE-3433
>             Project: HBase
>          Issue Type: Improvement
>          Components: performance, regionserver
>            Reporter: stack
>            Priority: Critical
>             Fix For: 0.92.0
> Here is offending code from inside in StoreScanner#next:
> {code}
>       // kv is no longer immutable due to KeyOnlyFilter! use copy for safety
>       KeyValue copyKv = new KeyValue(kv.getBuffer(), kv.getOffset(), kv.getLength());
> {code}
> This looks wrong given philosophy up to this has been avoidance of garbage-making copies.
> Maybe this has been looked into before and this is the only thing to be done but why
is KeyOnlyFilter not making copies rather than mutating originals?
> Making this critical against 0.92.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message