hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11208) Remove the hbase.hstore.blockingStoreFiles setting
Date Tue, 05 Aug 2014 17:42:13 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14086523#comment-14086523

Lars Hofhansl commented on HBASE-11208:

There are two issues: OOMs and read performance.

We'd then need another way than to "configure" HBase for read vs. write loads. This setting
is what puts an upper bound on acceptable read performance by limiting the number of HFiles
a scan has to look through - at the expense of blocking writers if they write faster than
the IO subsystem can absorb. The key is some mechanism to force HBase to slow clients down
or block them completely from writing if some "guaranteed" read performance is desired.

> Remove the hbase.hstore.blockingStoreFiles setting
> --------------------------------------------------
>                 Key: HBASE-11208
>                 URL: https://issues.apache.org/jira/browse/HBASE-11208
>             Project: HBase
>          Issue Type: Brainstorming
>          Components: Compaction, regionserver
>    Affects Versions: 0.99.0
>            Reporter: Nicolas Liochon
>            Assignee: Nicolas Liochon
>             Fix For: 0.99.0
> It's a little bit of a provocation, but the rational is:
>  - there are some bugs around the delayed flush. For example, if the periodic scheduler
has asked for a delayed flush, and that we need to flush, we will have to wait
>  - if the number of WAL files increases, we won't flush immediately if the blockingFile
number has been reached. This impacts the MTTR.
>  - We don't write to limit the compaction impact, but they are many cases where we would
want to flush anyway, as the writes cannot wait.
>  - this obviously leads to huge write latency peaks.
> So I'm questioning this setting, it leads to multiple intricate cases, unpredictable
write latency, and looks like a workaround for compaction performances. With all the work
done on compaction, I think we can get rid of it.  A solution in the middle would be to deprecate
it and to set it to a large value...
> Any opinion before I shoot :-) ? 

This message was sent by Atlassian JIRA

View raw message