hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zee Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13259) mmap() based BucketCache IOEngine
Date Thu, 23 Apr 2015 21:49:40 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14509896#comment-14509896
] 

Zee Chen commented on HBASE-13259:
----------------------------------

[~ndimiduk]  I have run the comparison between mmap and fileio and mmap and offheap quite
a few times. My fuzzy memory of observations from 2 months ago were:
- mmap is identical to offheap in both latency and throughput
- mmap is slightly better than file in latency but it has greater throughput

Since offheap engine also allocates memory using mmap(), except using anonymous maps as opposed
to file backed map, the identical performance result is not particularly surprising. In both
cases the bulk of the cost is in copying the data from mmap areas and we avoid a trip to the
kernel.

I haven't tested this with those cache options you mentioned yet. I did test with a few different
block sizes and bucket sizes, but haven't found any significant impact on latency yet.


> mmap() based BucketCache IOEngine
> ---------------------------------
>
>                 Key: HBASE-13259
>                 URL: https://issues.apache.org/jira/browse/HBASE-13259
>             Project: HBase
>          Issue Type: New Feature
>          Components: BlockCache
>    Affects Versions: 0.98.10
>            Reporter: Zee Chen
>             Fix For: 2.0.0, 1.2.0
>
>         Attachments: HBASE-13259-v2.patch, HBASE-13259.patch, ioread-1.svg, mmap-0.98-v1.patch,
mmap-1.svg, mmap-trunk-v1.patch
>
>
> Of the existing BucketCache IOEngines, FileIOEngine uses pread() to copy data from kernel
space to user space. This is a good choice when the total working set size is much bigger
than the available RAM and the latency is dominated by IO access. However, when the entire
working set is small enough to fit in the RAM, using mmap() (and subsequent memcpy()) to move
data from kernel space to user space is faster. I have run some short keyval gets tests and
the results indicate a reduction of 2%-7% of kernel CPU on my system, depending on the load.
On the gets, the latency histograms from mmap() are identical to those from pread(), but peak
throughput is close to 40% higher.
> This patch modifies ByteByfferArray to allow it to specify a backing file.
> Example for using this feature: set  hbase.bucketcache.ioengine to mmap:/dev/shm/bucketcache.0
in hbase-site.xml.
> Attached perf measured CPU usage breakdown in flames graph.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message