hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erik Holstad (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1192) LRU-style map for the block cache
Date Mon, 09 Feb 2009 19:00:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671960#action_12671960

Erik Holstad commented on HBASE-1192:

In the instructions for SoftReferences you can find:
"All soft references to softly-reachable objects are guaranteed to have been cleared before
the virtual machine throws an OutOfMemoryError. Otherwise no constraints are placed upon the
time at which a soft reference will be cleared or the order in which a set of such references
to different objects will be cleared. Virtual machine implementations are, however, encouraged
to bias against clearing recently-created or recently-used soft references."

Which would mean that the behavior would change from JVM to JVM, so the Lruness of the system
is clearly questionable.

Ran a test where I used SoftSortedValueMap and inserted 10M entries into it, all entries were
the same size.
i is the entry currently inserted

size of map before 1285540
size of deletes 1285537
size of map after 3
i 1285540

size of map before 1287469
size of deletes 443
size of map after 1287026
i 2573006

size of map before 1287027
size of deletes 2075
size of map after 1284952
i 2573007

size of map before 1284953
size of deletes 692
size of map after 1284261
i 2573008

size of map before 1284262
size of deletes 624
size of map after 1283638
i 2573009

size of map before 1283639
size of deletes 650
size of map after 1282989
i 2573010

size of map before 1282990
size of deletes 672
size of map after 1282318
i 2573011

size of map before 1282319
size of deletes 632
size of map after 1281687
i 2573012

size of map before 1281688
size of deletes 1281679
size of map after 9
i 2573013

size of map before 1285424
size of deletes 1285423
size of map after 1
i 3858428

size of map before 1286746
size of deletes 1286745
size of map after 1
i 5145173

size of map before 1285660
size of deletes 1285659
size of map after 1
i 6430832

size of map before 1286804
size of deletes 1286802
size of map after 2
i 7717635

size of map before 1286745
size of deletes 1286744
size of map after 1
i 9004378

So one can see that sometimes it evicts a some 100 entries for every insert and other times
it basically empties the whole map, but waits long in between.

> LRU-style map for the block cache
> ---------------------------------
>                 Key: HBASE-1192
>                 URL: https://issues.apache.org/jira/browse/HBASE-1192
>             Project: Hadoop HBase
>          Issue Type: New Feature
>          Components: regionserver
>            Reporter: Jonathan Gray
>            Priority: Blocker
>             Fix For: 0.20.0
> We need to decide what structure to use to back the block cache.  The primary decision
is whether to continue using SoftReferences or to build our own structure.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message