Return-Path: X-Original-To: apmail-hbase-dev-archive@www.apache.org Delivered-To: apmail-hbase-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 66A69D6A0 for ; Thu, 7 Mar 2013 22:56:55 +0000 (UTC) Received: (qmail 3247 invoked by uid 500); 7 Mar 2013 22:56:54 -0000 Delivered-To: apmail-hbase-dev-archive@hbase.apache.org Received: (qmail 3150 invoked by uid 500); 7 Mar 2013 22:56:54 -0000 Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hbase.apache.org Delivered-To: mailing list dev@hbase.apache.org Received: (qmail 3142 invoked by uid 99); 7 Mar 2013 22:56:54 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Mar 2013 22:56:54 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of saint.ack@gmail.com designates 209.85.214.42 as permitted sender) Received: from [209.85.214.42] (HELO mail-bk0-f42.google.com) (209.85.214.42) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 07 Mar 2013 22:56:47 +0000 Received: by mail-bk0-f42.google.com with SMTP id jk7so472286bkc.1 for ; Thu, 07 Mar 2013 14:56:27 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:x-received:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; bh=kLnHmTKdGUTluQHav3IBcR4G1srQU5YHBu2Fio/3KUc=; b=Qauh5P6BmsgBJe6kcSqzsEkvq9c39iAyoZCIMAPyBJwmF9dSczvlWe72Qhw++WRsyN Bmz7Pj1PYAatheaRh6+D3wVcgQydfyt0OLdJM32wRP0ucHI9ysf8mXaZI7rBoqkrG9KB LfRA0MwJCUip6/td9QMHkZOU6cXmdwHxvHl0+4ByvriRmBP0WD3PS4SsMajv6sGkjdhm WQD/uXP6QrAPfKu7Cu/wpLwsY20Jm5p+0GzLx0OliE7ltyJas+VP3Cz2dY9LKeH0CUoj dziZpvKjJuOC3a2A/fNlSW0M6a3yvketW+mXXo3jq5UfCAM7Ce16GrZJy1GhZvQEREkC 0PGQ== MIME-Version: 1.0 X-Received: by 10.205.35.72 with SMTP id sv8mr27174bkb.24.1362696987193; Thu, 07 Mar 2013 14:56:27 -0800 (PST) Sender: saint.ack@gmail.com Received: by 10.204.147.22 with HTTP; Thu, 7 Mar 2013 14:56:27 -0800 (PST) In-Reply-To: References: Date: Thu, 7 Mar 2013 14:56:27 -0800 X-Google-Sender-Auth: QwD-DB85krmNYMbzCKrnjeIFyfM Message-ID: Subject: Re: reason to do major compaction after split From: Stack To: HBase Dev List Content-Type: multipart/alternative; boundary=bcaec52c68ffdbe61504d75d9d59 X-Virus-Checked: Checked by ClamAV on apache.org --bcaec52c68ffdbe61504d75d9d59 Content-Type: text/plain; charset=UTF-8 On Thu, Mar 7, 2013 at 2:28 PM, Matteo Bertozzi wrote: > This is seems to going in a super messy direction. > Smile. I was thinking you'd show up on this thread. Agree. > With HBASE-7806 the ideas was to cleanup all this crazy stuff (HFileLink, > References, ...) > > unfortunately the initial decision of tight together the fs layout > and the tables/regions/families is bringing to all this workaround to have > something cool. > > If you put the files in one place, and the association in another you can > avoid all this complexity. > > /hbase/data/[file1, file 2, file 3, file N] > > table 1/region 1: [file 2] > table 1/region 2: [file 1 (from 0 to 50)] > table 1/region 3: [file 1 (from 50 to 100)] > table 2/region 1: [file 1, file 2] > > Any ideas on what migration from current format to the above would be like Matteo? We'd read current layout, use it to populate a files table, new files would be written to a the new /hbase/data/ dir, and for a while we'd span the old and new locations? St.Ack --bcaec52c68ffdbe61504d75d9d59--