hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Todd Lipcon <t...@cloudera.com>
Subject Re: Heap fragmentation
Date Thu, 20 Jan 2011 20:20:08 GMT
Yep, the difficulty is dangling references that leak from Memstore out to
other parts of the system. EG keyvalueheaps in scanners, IPC responses
trying to achieve zero-copy. Reference counting is possible but not trivial.

On Thu, Jan 20, 2011 at 12:18 PM, Jay Booth <jaybooth@gmail.com> wrote:

> Do you even need to go unsafe?  Say you made the memstore a
> closed-loop system..  get allocated 512MB or 1GB of buffers at
> startup, and then whenever any buffer is flushed, it gets pushed onto
> the bufferQueue.  New buffers only come from bufferQueue, no new
> byte[] calls.  If there are no buffers available in the queue, and you
> need to write, trigger a flush.  That means it'll be the same buffers,
> which won't create new heap fragmentation as they'll presumably
> migrate down to the bottom of old gen heap after the first major
> compaction or two..  you're just passing references to them through
> that queue.  I haven't look at the code but this is probably
> reasonably pluggable with whatever safeguards you guys already have to
> flush before allocing if you're close to max memory.
>
> On Thu, Jan 20, 2011 at 3:14 PM, Ryan Rawson <ryanobjc@gmail.com> wrote:
> > Bigmemory serializes java objects then stores the serialized form in
> > DirectByteBuffers, which isn't quite exactly what we want...
> >
> > Needless to say a number of us have been poking at the DBB and Unsafe
> > APIs and have a few ideas :-)
> >
> > On Thu, Jan 20, 2011 at 10:39 AM, Todd Lipcon <todd@cloudera.com> wrote:
> >> On Thu, Jan 20, 2011 at 10:34 AM, Vladimir Rodionov <
> vrodionov@carrieriq.com
> >>> wrote:
> >>
> >>> BigMemory anyone?
> >>>
> >>> http://www.terracotta.org/bigmemory
> >>>
> >>>
> >> Yea, something like that, minus the commercial license ;-)
> >>
> >> -Todd
> >> ____
> >>
> >>> From: Todd Lipcon [todd@cloudera.com]
> >>> Sent: Thursday, January 20, 2011 9:33 AM
> >>> To: dev
> >>> Subject: Heap fragmentation
> >>>
> >>> I did some experiments to understand our full GC issues better last
> night.
> >>> Here are the results:
> >>> http://people.apache.org/~todd/hbase-fragmentation/
> >>>
> >>> <http://people.apache.org/~todd/hbase-fragmentation/>Basically my
> >>> conclusion
> >>> here is that (for these YCSB workloads) the memstore is way worse for
> heap
> >>> fragmentation than the block cache.
> >>>
> >>> Also we now have some tools and reference for comparison if we make any
> >>> changes to memstore to try to combat this issue.
> >>>
> >>> -Todd
> >>> --
> >>> Todd Lipcon
> >>> Software Engineer, Cloudera
> >>>
> >>
> >>
> >>
> >> --
> >> Todd Lipcon
> >> Software Engineer, Cloudera
> >>
> >
>



-- 
Todd Lipcon
Software Engineer, Cloudera

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message