Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 58379CE76 for ; Thu, 20 Jun 2013 02:08:23 +0000 (UTC) Received: (qmail 10568 invoked by uid 500); 20 Jun 2013 02:08:23 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 10536 invoked by uid 500); 20 Jun 2013 02:08:23 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 10527 invoked by uid 99); 20 Jun 2013 02:08:23 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 20 Jun 2013 02:08:23 +0000 Date: Thu, 20 Jun 2013 02:08:23 +0000 (UTC) From: "Sergey Shelukhin (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HBASE-8541) implement flush-into-stripes in stripe compactions MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-8541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HBASE-8541: ------------------------------------ Attachment: HBASE-8541-latest-with-dependencies.patch HBASE-8541-v1.patch > implement flush-into-stripes in stripe compactions > -------------------------------------------------- > > Key: HBASE-8541 > URL: https://issues.apache.org/jira/browse/HBASE-8541 > Project: HBase > Issue Type: Improvement > Reporter: Sergey Shelukhin > Attachments: HBASE-8541-latest-with-dependencies.patch, HBASE-8541-latest-with-dependencies.patch, HBASE-8541-v0.patch, HBASE-8541-v1.patch > > > Flush will be able to flush into multiple files under this design, avoiding L0 I/O amplification. > I have the patch which is missing just one feature - support for concurrent flushes and stripe changes. This can be done via extensive try-locking of stripe changes and flushes, or advisory flags without blocking flushes, dumping conflicting flushes into L0 in case of (very rare) collisions. For file loading for the latter, a set-cover-like problem needs to be solved to determine optimal stripes. That will also address Jimmy's concern of getting rid of metadata, btw. However currently I don't have time for that. I plan to attach the try-locking patch first, but this won't happen for a couple weeks probably and should not block main reviews. Hopefully this will be added on top of main reviews. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira