Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Sun, 26 Apr 2015 18:31:38 +0000 (UTC)
From: "Nick Dimiduk (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12782434.1426555241000.31779.1430073098559@Atlassian.JIRA>
In-Reply-To: <JIRA.12782434.1426555241000@Atlassian.JIRA>
References: <JIRA.12782434.1426555241000@Atlassian.JIRA>
 <JIRA.12782434.1426555241198@arcas>
Subject: [jira] [Commented] (HBASE-13259) mmap() based BucketCache IOEngine
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-13259?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14513177#comment-14513177 ] 

Nick Dimiduk commented on HBASE-13259:
--------------------------------------

Alright. It's too late now for 1.1, but we should be able to get this for 1.2. [~stack] what other tests would you like to see run? I can see about finding some cycles in a couple weeks.

> mmap() based BucketCache IOEngine
> ---------------------------------
>
>                 Key: HBASE-13259
>                 URL: https://issues.apache.org/jira/browse/HBASE-13259
>             Project: HBase
>          Issue Type: New Feature
>          Components: BlockCache
>    Affects Versions: 0.98.10
>            Reporter: Zee Chen
>             Fix For: 2.0.0, 1.2.0
>
>         Attachments: HBASE-13259-v2.patch, HBASE-13259.patch, ioread-1.svg, mmap-0.98-v1.patch, mmap-1.svg, mmap-trunk-v1.patch
>
>
> Of the existing BucketCache IOEngines, FileIOEngine uses pread() to copy data from kernel space to user space. This is a good choice when the total working set size is much bigger than the available RAM and the latency is dominated by IO access. However, when the entire working set is small enough to fit in the RAM, using mmap() (and subsequent memcpy()) to move data from kernel space to user space is faster. I have run some short keyval gets tests and the results indicate a reduction of 2%-7% of kernel CPU on my system, depending on the load. On the gets, the latency histograms from mmap() are identical to those from pread(), but peak throughput is close to 40% higher.
> This patch modifies ByteByfferArray to allow it to specify a backing file.
> Example for using this feature: set  hbase.bucketcache.ioengine to mmap:/dev/shm/bucketcache.0 in hbase-site.xml.
> Attached perf measured CPU usage breakdown in flames graph.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)