hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Appy (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-16554) Procedure V2 - Recover 'updated' part of WAL tracker if trailer is corrupted.
Date Mon, 19 Sep 2016 19:30:21 GMT

     [ https://issues.apache.org/jira/browse/HBASE-16554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Appy updated HBASE-16554:
-------------------------
    Description: 
If the last wal was closed cleanly, the global tracker will be the last wal tracker (no rebuild
needed)
if the last wal does not have a tracker (corrupted/master-killed). on load() we will rebuild
the global tracker.

To compute quickly which files should be deleted, we also want the tracker of each file.
if the wal was closed properly and has a tracker we are good, if not we need to rebuild the
tracker for that file.
each file tracker contains a bitmap about what is in the wal (the updated bitmap), which is
easy to compute just by reading each entry of the wal.
The 'deleted' bitmap keeps track of the "running procedures" up to that wal, however, it's
not required by WAL cleaner, so we don't bother recovering it.

  was:
If the last wal was closed cleanly, the global tracker will be the last wal tracker (no rebuild
needed)
if the last wal does not have a tracker (corrupted/master-killed). on load() we will rebuild
the global tracker.

To compute quickly which files should be deleted, we also want the tracker of each file.
if the wal was closed properly and has a tracker we are good, if not we need to rebuild the
tracker for that file.
each file tracker contains a bitmap about what is in the wal (the updated bitmap), which is
easy to compute just by reading each entry of the wal.


> Procedure V2 - Recover 'updated' part of WAL tracker if trailer is corrupted.
> -----------------------------------------------------------------------------
>
>                 Key: HBASE-16554
>                 URL: https://issues.apache.org/jira/browse/HBASE-16554
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Appy
>            Assignee: Appy
>             Fix For: 2.0.0
>
>         Attachments: HBASE-16554.master.001.patch, HBASE-16554.master.002.patch, tracker-rebuild.patch
>
>
> If the last wal was closed cleanly, the global tracker will be the last wal tracker (no
rebuild needed)
> if the last wal does not have a tracker (corrupted/master-killed). on load() we will
rebuild the global tracker.
> To compute quickly which files should be deleted, we also want the tracker of each file.
> if the wal was closed properly and has a tracker we are good, if not we need to rebuild
the tracker for that file.
> each file tracker contains a bitmap about what is in the wal (the updated bitmap), which
is easy to compute just by reading each entry of the wal.
> The 'deleted' bitmap keeps track of the "running procedures" up to that wal, however,
it's not required by WAL cleaner, so we don't bother recovering it.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message