Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 2 Sep 2016 22:09:20 +0000 (UTC)
From: "Matteo Bertozzi (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.13002485.1472850727000.483218.1472854160697@Atlassian.JIRA>
In-Reply-To: <JIRA.13002485.1472850727000@Atlassian.JIRA>
References: <JIRA.13002485.1472850727000@Atlassian.JIRA> <JIRA.13002485.1472850727379@arcas>
Subject: [jira] [Updated] (HBASE-16554) Procedure V2 - handle corruption of
 WAL trailer
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 02 Sep 2016 22:09:22 -0000


     [ https://issues.apache.org/jira/browse/HBASE-16554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Matteo Bertozzi updated HBASE-16554:
------------------------------------
    Description: 
If the last wal was closed cleanly, the global tracker will be the last wal tracker (no rebuild needed)
if the last wal does not have a tracker (corrupted/master-killed). on load() we will rebuild the global tracker.

To compute quickly which files should be deleted, we also want the tracker of each file.
if the wal was closed properly and has a tracker we are good, if not we need to rebuild the tracker for that file.
each file tracker contains a bitmap about what is in the wal (the updated bitmap), which is easy to compute just by reading each entry of the wal. and a bitmap that keeps track of the "running procedures" up to that wal (the deleted bitmap). The delete bitmap requires a bit of post read-all-wals work. and it will basically require to AND the deleted bitmap of 

  was:if trailer is corrupted, build tracker for each WAL when loading, then use these trackers to build global tracker.


> Procedure V2 - handle corruption of WAL trailer
> -----------------------------------------------
>
>                 Key: HBASE-16554
>                 URL: https://issues.apache.org/jira/browse/HBASE-16554
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Appy
>            Assignee: Appy
>         Attachments: tracker-rebuild.patch
>
>
> If the last wal was closed cleanly, the global tracker will be the last wal tracker (no rebuild needed)
> if the last wal does not have a tracker (corrupted/master-killed). on load() we will rebuild the global tracker.
> To compute quickly which files should be deleted, we also want the tracker of each file.
> if the wal was closed properly and has a tracker we are good, if not we need to rebuild the tracker for that file.
> each file tracker contains a bitmap about what is in the wal (the updated bitmap), which is easy to compute just by reading each entry of the wal. and a bitmap that keeps track of the "running procedures" up to that wal (the deleted bitmap). The delete bitmap requires a bit of post read-all-wals work. and it will basically require to AND the deleted bitmap of 


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)