cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yang Yang (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (CASSANDRA-2252) arena allocation for memtables
Date Thu, 25 Aug 2011 19:10:31 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-2252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13091225#comment-13091225
] 

Yang Yang edited comment on CASSANDRA-2252 at 8/25/11 7:09 PM:
---------------------------------------------------------------

my line of thought comes from update-heavy workload (counters etc). conceivably (putting the
row key issue aside for a while)  one region would contain bytebuffer values of similar age.
as more updates come in, all the columns in older regions are likely to have all died out,
thus allowing us to free the entire region before flushing happens.  


coming back to the row key issue, in the original slab allocator paper ( Jeff Bonwick ) ,
a slab contains strictly the same objects, which imply that they die at roughly the same time.
  if they don't, then yes, in our case, slab has the disadvantage that an entire slab (2MB
worth of mem) is held simply because a row key in it is not dead yet.  so to overcome this
disadvantage, we probably need to further distinguish between object types to be allocated
in the slab: this JIRA (same as HBase code) distinguishes between all the allocations between
different memtables, to work better with update-heavy traffic, we need to *distinguish between
row keys and column values (they have different life times)*




      was (Author: yangyangyyy):
    my line of thought comes from update-heavy workload (counters etc). conceivably (putting
the row key issue aside for a while)  one region would contain bytebuffer values of similar
age. as more updates come in, all the columns in older regions are likely to have all died
out, thus allowing us to free the entire region before flushing happens.  


coming back to the row key issue, in the original slab allocator paper ( Jeff Bonwick ) ,
a slab contains strictly the same objects, which imply that they die at roughly the same time.
  if they don't, then yes, in our case, slab has the disadvantage that an entire slab (2MB
worth of mem) is held simply because a row key in it is not dead yet.  so to overcome this
disadvantage, we probably need to further distinguish between object types to be allocated
in the slab: this JIRA (same as HBase code) distinguishes between all the allocations between
different memtables, to work better with update-heavy traffic, we need to distinguish between
row keys and column values (they have different life times)



  
> arena allocation for memtables
> ------------------------------
>
>                 Key: CASSANDRA-2252
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2252
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Jonathan Ellis
>            Assignee: Jonathan Ellis
>             Fix For: 1.0
>
>         Attachments: 0001-add-MemtableAllocator.txt, 0002-add-off-heap-MemtableAllocator-support.txt,
2252-v3.txt, 2252-v4.txt, merged-2252.tgz
>
>
> The memtable design practically actively fights Java's GC design.  Todd Lipcon gave a
good explanation over on HBASE-3455.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message