hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HBASE-4583) Integrate RWCC with Append and Increment operations
Date Mon, 29 Oct 2012 21:16:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13486363#comment-13486363
] 

Lars Hofhansl edited comment on HBASE-4583 at 10/29/12 9:15 PM:
----------------------------------------------------------------

Yes, I think the same can work for checkAndXXX... In fact maybe now we can unify all of those.
They all have 3 steps:
# precondition/get old values
# internal logic
# update values

It might be possible to extract that boilerplate and pass in an implementation of an "AtomicOperation"
interface or something.

In terms of performance... It will be slower. It's no longer doing upserts (i.e. removing
old KVs still found in the memstore).
Personally I do not understand why Increment would be special here. All other operations work
by creating new versions and they all could be sped up if we would remove stale versions from
the memstore immediately. Why are Puts not doing upserts? And why would we care less about
the history of an increment column?
I think if we wanted an upsert type functionality it should be an option for every operation
(except Delete, I guess). We'd declare we do not care about the history and then the operation
could go through existing versions of KVs for which it can prove that no scanner is using
them (by using the region's smallest readpoint).

The only performance test I did was to run TestAtomicOperation#testIncrementMultiThreads,
which is about 30% slower (6.5s vs 5s before). This is in part due to the MVCC enforcement
and in part due to the fact the memstore will fill up sooner.

Edit: Spelling
                
      was (Author: lhofhansl):
    Yes, I think the same can work for checkAndXXX... In fact maybe now we can unify all of
those. They all have 3 steps:
# precondition/get old values
# internal logic
# update values

It might be possible to extract that boilerplate and pass in an implementation of an "AtomicOperation"
interface or something.

In terms of performance... It will be slower. It's no longer doing upserts (i.e. removing
old KVs still found in the memstore).
Personally I do not understand by Increment would be special here. All other operations work
by creating new versions and they all could be sped up if we would remove stale versions from
the memstore immediately. Why are Puts not doing upserts? And why would we care less about
the history of an increment column?
I think if we wanted an upsert type functionality it should be an option for every operation
(except Delete, I guess). We'd declare we do not care about the history and then the operation
could go through existing versions of KVs for which it can prove that no scanner is using
them (by using the region's smallest readpoint).

The only performance test I did was to run TestAtomicOperation#testIncrementMultiThreads,
which is about 30% slower (6.5s vs 5s before). This is in part due to the MVCC enforcement
and in part due to the fact the memstore will fill up sooner.
                  
> Integrate RWCC with Append and Increment operations
> ---------------------------------------------------
>
>                 Key: HBASE-4583
>                 URL: https://issues.apache.org/jira/browse/HBASE-4583
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Lars Hofhansl
>            Assignee: Lars Hofhansl
>             Fix For: 0.94.3, 0.96.0
>
>         Attachments: 4583-trunk-radical.txt, 4583-trunk-radical_v2.txt, 4583-trunk-v3.txt,
4583.txt, 4583-v2.txt, 4583-v3.txt, 4583-v4.txt
>
>
> Currently Increment and Append operations do not work with RWCC and hence a client could
see the results of multiple such operation mixed in the same Get/Scan.
> The semantics might be a bit more interesting here as upsert adds and removes to and
from the memstore.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message