hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-18133) Low-latency space quota size reports
Date Fri, 16 Jun 2017 02:28:01 GMT

     [ https://issues.apache.org/jira/browse/HBASE-18133?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Josh Elser updated HBASE-18133:
    Attachment: HBASE-18133.001.patch

.001 which was a nice little change in the end.

Still requires HBASE-17752 and HBASE-17840 to land.

> Low-latency space quota size reports
> ------------------------------------
>                 Key: HBASE-18133
>                 URL: https://issues.apache.org/jira/browse/HBASE-18133
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>             Fix For: 3.0.0
>         Attachments: HBASE-18133.001.patch
> Presently space quota enforcement relies on RegionServers sending reports to the master
about each Region that they host. This is done by periodically, reading the cached size of
each HFile in each Region (which was ultimately computed from HDFS).
> This means that the Master is unaware of Region size growth until the the next time this
chore in a RegionServer fires which is a fair amount of latency (a few minutes, by default).
Operations like flushes, compactions, and bulk-loads are delayed even though the RegionServer
is running those operations locally.
> Instead, we can create an API which these operations could invoke that would automatically
update the size of the Region being operated on. For example, a successful flush can report
that the size of a Region increased by the size of the flush. A compaction can subtract the
size of the input files of the compaction and add in the size of the resulting file.
> This de-couples the computation of a Region's size from sending the Region sizes to the
Master, allowing us to send reports more frequently, increasing the responsiveness of the
cluster to size changes.

This message was sent by Atlassian JIRA

View raw message