Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 4 Nov 2015 03:18:27 +0000 (UTC)
From: "Andrew Purtell (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12910233.1446606287000.146123.1446607107690@Atlassian.JIRA>
In-Reply-To: <JIRA.12910233.1446606287000@Atlassian.JIRA>
References: <JIRA.12910233.1446606287000@Atlassian.JIRA>
 <JIRA.12910233.1446606287813@arcas>
Subject: [jira] [Commented] (HBASE-14757) Reduce allocation pressure imposed
 by HFile block processing
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-14757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14988808#comment-14988808 ] 

Andrew Purtell commented on HBASE-14757:
----------------------------------------

For HFileBlock read buffers: These allocations are expected. This is where block data lands. These allocations are carried through into the block cache when using the LRUBlockCache and are retained potentially for a very long time. When using BucketCache, the on-disk block data isn't needed after the IO engine serializes it off-heap or to SSD. We could reuse allocations for the latter case but not the former. Needs investigation. 

Meanwhile, for block encoding / seeker state, I think these are all short lived allocations. So, a strawman: Round up byte[] allocation requests here to the nearest power of 2. Keep object caches for powers of 2 over a common range. Serve requests from the appropriate free list where available. Prune free lists using LRU. 

> Reduce allocation pressure imposed by HFile block processing
> ------------------------------------------------------------
>
>                 Key: HBASE-14757
>                 URL: https://issues.apache.org/jira/browse/HBASE-14757
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Andrew Purtell
>            Assignee: Andrew Purtell
>             Fix For: 2.0.0, 1.2.0, 1.3.0, 0.98.17
>
>
> Using Flight Recorder to look at the object allocation profiles of regionservers processing the various YCSB workloads when block encoding is enabled (specifically, FAST_DIFF, but this applies to any), we can see:
> - Allocations of byte[] for block encoding contribute 40-70% of all allocation pressure in TLABs. 
> - Of that subset of allocation pressure, ~50-70% is byte[] for SeekerState
> - Greater than 99% of allocation of byte[] outside of TLABs are for read buffers for HFileBlock#readBlockDataInternal.
> This issue is for investigation of strategy for and impact of reducing that allocation pressure. Reducing allocation pressure reduces demand for GC, which reduces GC activity overall, which reduces a source of system latency.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)