hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: ConcurrentSkipListMap performance in the MemStore
Date Tue, 14 Jun 2011 22:51:44 GMT
Or do as Todd suggested a while back and pull in CSLM and hack it to
suit our needs; its public domain.
St.Ack

On Tue, Jun 14, 2011 at 3:23 PM, Jason Rutherglen
<jason.rutherglen@gmail.com> wrote:
>> See this J: http://www.cloudera.com/blog/2011/03/avoiding-full-gcs-in-hbase-with-memstore-local-allocation-buffers-part-3/
>
> Interesting.
>
> In Lucene for realtime search we have the same problem where the
> best/only option right now for the realtime terms dictionary is to use
> ConcurrentSkipListMap.  However the Node, key, value, and Index
> objects all are pointers that consume RAM and need to be cleaned up.
>
> Lucene right now during indexing does not generate much object pointer
> garbage.  We have the same problem in the two projects.
>
> Maybe there's a way to create an AtomicIntegerArray based system that
> uses int pointers to perform the same function as a the concurrent
> skip list.  It would greatly reduce garbage generation.
>
> On Tue, Jun 14, 2011 at 10:14 AM, Stack <stack@duboce.net> wrote:
>> See this J: http://www.cloudera.com/blog/2011/03/avoiding-full-gcs-in-hbase-with-memstore-local-allocation-buffers-part-3/
>> St.Ack
>>
>> On Mon, Jun 13, 2011 at 10:58 PM, Jason Rutherglen
>> <jason.rutherglen@gmail.com> wrote:
>>> Is there an issue with garbage generated from using CSLM in the MemStore?
>>>
>>
>

Mime
View raw message