hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron Tokhy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15314) Allow more than one backing file in bucketcache
Date Thu, 26 Jan 2017 23:08:24 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15314?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840635#comment-15840635

Aaron Tokhy commented on HBASE-15314:

bq. 1. It will take an extra IO if an allocation cross files, but the probability is about
32KB/320GB ≈ 1/10M ( example for block size 32KB, one file capacity 320GB). So, I think
it's no effect for perfomance

The performance difference with either approach is negligible as you mentioned.  The guarantee
is that an allocation is guaranteed to reside on a single file.

bq. 2. With crossing files allocation, no extra logic is needed if a single file fails due
to an IO error. We will free the whole alloction if failed (see this logic from BucketCahe#writeToCache).

Yes this would then require no additional changes in BucketCache.

bq. IMPO, make things simple if no obvious benefit, I prefer to use the thought of 1st patch.

My patch employs almost the exact same approach as [~anoop.hbase]'s (the first patch) and
employs the work [~Amal Joshy] added to avoid crossing file boundaries, while adding plenty
of tests.

There are a couple of things you can gain by ensuring that HBase blocks are guaranteed to
reside in a single segment:

- Turning off files in a JBOD configuration when an IOException occurs.  Only the allocations
that resided in the segment that resided on the device that failed can be invalided, and the
BucketCache can continue.  This does not happen in the patch I submitted, but it could be
an improvement that you wouldn't be able to easily handle if HBase blocks crossed segments.

- Implementing striding/striping without adding too much logic in BucketCache.  If a single
'RAID 0' volume fails (in a RAID 0 IOEngine decorator), only the allocations that resided
in one of the drives could be invalidated, and the BucketCache can still continue operating.

This could be added as a 'cache invalidation on device failure' feature for BucketCache.

> Allow more than one backing file in bucketcache
> -----------------------------------------------
>                 Key: HBASE-15314
>                 URL: https://issues.apache.org/jira/browse/HBASE-15314
>             Project: HBase
>          Issue Type: Sub-task
>          Components: BucketCache
>            Reporter: stack
>            Assignee: Aaron Tokhy
>         Attachments: FileIOEngine.java, HBASE-15314.master.001.patch, HBASE-15314.master.001.patch,
HBASE-15314.patch, HBASE-15314-v2.patch, HBASE-15314-v3.patch
> Allow bucketcache use more than just one backing file: e.g. chassis has more than one
SSD in it.

This message was sent by Atlassian JIRA

View raw message