hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: optimizing for random access
Date Tue, 27 Apr 2010 00:10:13 GMT
One other issue we haven't talked about is JBOD systems - some people
run over a dozen disks per machine. I run 4 disks/node. HDFS does a
valuable service for HBase in balancing IO across multiple JBODs on a
single node.  Handling disk issues is something we can completely
ignore in HBase right now, it would be nice to keep it that way.

On Mon, Apr 26, 2010 at 4:52 PM, Stack <stack@duboce.net> wrote:
> On Mon, Apr 26, 2010 at 3:36 PM, Geoff Hendrey <ghendrey@decarta.com> wrote:
>> My thought with memory mapping was, as you noted, *not* to try to map files that
are inside of HDFS but rather to copy as many blocks as possible out of HDFS, onto region
server filesystems, and memory map the file on the region server. TB drives are now common.
The virtual memory system of the Operating System manages paging in and out of "real" memory
off disk when you use memory mapping. My experience with memory mapped ByteBuffer in Java
is that it is very fast and scalable. By fast, I mean I have clocked reads in the microseconds
using nanotime. So I was just wondering why you wouldn't at least make a 2nd level cache with
memory mapping.
> Are memory-mapped files scalable in java?  I'm curious.  Its a while
> since I played with them (circa java 1.5) but then they did not scale.
> I was only able to open a few files concurrently before I started
> running into "interesting" issue.   In hbase I'd need to be able to
> keep hundreds or even thousands open concurrently.
> I've thought about doing something like you propose Geoff -- keeping
> some subset of storefiles locally (we could even write two places when
> compacting say, local and out to hdfs) -- but it always devolved
> quickly into a complicated mess keeping up the local copy with the
> remote set making sure the local didn't overflow local storage and
> that local files were aged out on compactions and splits.  If you have
> suggestion on how it'd work, I'd all ears.
> Thanks,
> St.Ack

View raw message