hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Helmling (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16755) Honor flush policy under global memstore pressure
Date Fri, 03 Mar 2017 17:59:45 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15894773#comment-15894773
] 

Gary Helmling commented on HBASE-16755:
---------------------------------------

bq. This seems like an improvement, and not a bug. And the risk is non-trivial as per above.
So, I would say it should not go to 1.3.1 anyway. It can be 1.4, and 2.0.

I would argue it is a bug that we're not respecting the configured flush policy under global
memstore pressure.  And the performance degradation that can occur with tables with multiple
column family with divergent write rates can be pretty severe.  I don't know why we wouldn't
want to continue to allow FlushLargeStoresPolicy to choose the right stores to flush in this
situation.  It does still fall back to all stores in the case where no stores meet the flush
threshold.

If we don't fix this in 1.3.1, then we're not giving users any recourse if they run across
this, since even writing your own FlushPolicy can't override the behavior.

> Honor flush policy under global memstore pressure
> -------------------------------------------------
>
>                 Key: HBASE-16755
>                 URL: https://issues.apache.org/jira/browse/HBASE-16755
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: Ashu Pachauri
>            Assignee: Ashu Pachauri
>             Fix For: 1.3.1
>
>         Attachments: HBASE-16755.v0.patch
>
>
> When global memstore reaches the low water mark, we pick the best flushable region and
flush all column families for it. This is a suboptimal approach in the  sense that it leads
to an unnecessarily high file creation rate and IO amplification due to compactions. We should
still try to honor the underlying FlushPolicy.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message