hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward Bortnikov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-16417) In-Memory MemStore Policy for Flattening and Compactions
Date Thu, 27 Oct 2016 15:34:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-16417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15612232#comment-15612232

Edward Bortnikov commented on HBASE-16417:

[~anoop.hbase], [~eshcar] - the end state you both want to reach is the same, just the ways
of going there are different. Before going into any detail - the holy grail is a self-tuning
policy that does the right thing in EVERY use case of interest. We'd like to achieve it without
any additional configuration or private solutions for specific cases. Reason is - what ends
up as non-default option will never be used. 

Anoop - you want two things actually: (1) no compaction of any kind happen when there is no
redundancy and (2) flushing everything to disk when the memstore overflows. (Note that these
two can be decoupled.) Hopefully you don't mind if there's a policy that magically figures
out that we're in that use case, at practically zero cost, and does exactly (1) and (2). We
just disagree on marking the opposite case (many duplicates) as special and going down a different
code path there - because if we leave it to the admin as non-default we know what'll happen.

So we are after that magic policy. The quest won't take long but it has to be data-driven.
At the moment, we've just reproduced one microbenchmark (uniform writes, no reads), but there
are many other cases that should be looked at. We have the env to run them, and we'll be producing
those results over the next couple of weeks. We'll be very much transparent in the process,
publishing the results frequently. Once we have the data let's decide collectively. If nothing
universal we'll work we can always back off to configs but I'd consider that undesirable.

> In-Memory MemStore Policy for Flattening and Compactions
> --------------------------------------------------------
>                 Key: HBASE-16417
>                 URL: https://issues.apache.org/jira/browse/HBASE-16417
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Anastasia Braginsky
>            Assignee: Anastasia Braginsky
>             Fix For: 2.0.0

This message was sent by Atlassian JIRA

View raw message