hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Erik Holstad (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1192) LRU-style map for the block cache
Date Mon, 09 Feb 2009 19:00:59 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671960#action_12671960
] 

Erik Holstad commented on HBASE-1192:
-------------------------------------

In the instructions for SoftReferences you can find:
"All soft references to softly-reachable objects are guaranteed to have been cleared before
the virtual machine throws an OutOfMemoryError. Otherwise no constraints are placed upon the
time at which a soft reference will be cleared or the order in which a set of such references
to different objects will be cleared. Virtual machine implementations are, however, encouraged
to bias against clearing recently-created or recently-used soft references."

Which would mean that the behavior would change from JVM to JVM, so the Lruness of the system
is clearly questionable.


Ran a test where I used SoftSortedValueMap and inserted 10M entries into it, all entries were
the same size.
i is the entry currently inserted

size of map before 1285540
size of deletes 1285537
size of map after 3
i 1285540

size of map before 1287469
size of deletes 443
size of map after 1287026
i 2573006

size of map before 1287027
size of deletes 2075
size of map after 1284952
i 2573007

size of map before 1284953
size of deletes 692
size of map after 1284261
i 2573008

size of map before 1284262
size of deletes 624
size of map after 1283638
i 2573009

size of map before 1283639
size of deletes 650
size of map after 1282989
i 2573010

size of map before 1282990
size of deletes 672
size of map after 1282318
i 2573011

size of map before 1282319
size of deletes 632
size of map after 1281687
i 2573012

size of map before 1281688
size of deletes 1281679
size of map after 9
i 2573013

size of map before 1285424
size of deletes 1285423
size of map after 1
i 3858428

size of map before 1286746
size of deletes 1286745
size of map after 1
i 5145173

size of map before 1285660
size of deletes 1285659
size of map after 1
i 6430832

size of map before 1286804
size of deletes 1286802
size of map after 2
i 7717635

size of map before 1286745
size of deletes 1286744
size of map after 1
i 9004378

So one can see that sometimes it evicts a some 100 entries for every insert and other times
it basically empties the whole map, but waits long in between.

> LRU-style map for the block cache
> ---------------------------------
>
>                 Key: HBASE-1192
>                 URL: https://issues.apache.org/jira/browse/HBASE-1192
>             Project: Hadoop HBase
>          Issue Type: New Feature
>          Components: regionserver
>            Reporter: Jonathan Gray
>            Priority: Blocker
>             Fix For: 0.20.0
>
>
> We need to decide what structure to use to back the block cache.  The primary decision
is whether to continue using SoftReferences or to build our own structure.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message