Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 20 Jun 2013 02:01:21 +0000 (UTC)
From: "Sergey Shelukhin (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12647427.1368493860775.145730.1371693681137@arcas>
In-Reply-To: <JIRA.12647427.1368493860775@arcas>
References: <JIRA.12647427.1368493860775@arcas>
Subject: [jira] [Commented] (HBASE-8541) implement flush-into-stripes in
 stripe compactions
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-8541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13688789#comment-13688789 ] 

Sergey Shelukhin commented on HBASE-8541:
-----------------------------------------

{quote}
In code above, newStripes is put in filesForL0 - meaning we ignore conflictingFiles. But the error message says that newSize cannot be added. This seems to be inconsistent, right ?
{quote}
Conflicting files are old files sitting in the stripes. We ignore them so they just stay there; it's the new files that we put into L0.


> implement flush-into-stripes in stripe compactions
> --------------------------------------------------
>
>                 Key: HBASE-8541
>                 URL: https://issues.apache.org/jira/browse/HBASE-8541
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Sergey Shelukhin
>         Attachments: HBASE-8541-latest-with-dependencies.patch, HBASE-8541-v0.patch
>
>
> Flush will be able to flush into multiple files under this design, avoiding L0 I/O amplification.
> I have the patch which is missing just one feature - support for concurrent flushes and stripe changes. This can be done via extensive try-locking of stripe changes and flushes, or advisory flags without blocking flushes, dumping conflicting flushes into L0 in case of (very rare) collisions. For file loading for the latter, a set-cover-like problem needs to be solved to determine optimal stripes. That will also address Jimmy's concern of getting rid of metadata, btw. However currently I don't have time for that. I plan to attach the try-locking patch first, but this won't happen for a couple weeks probably and should not block main reviews. Hopefully this will be added on top of main reviews.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira