hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14135) HBase Backup/Restore Phase 3: Merge backup images
Date Thu, 03 Aug 2017 16:00:07 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14135?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16112993#comment-16112993

Josh Elser commented on HBASE-14135:

bq. This is not only for backup, right, Josh Elser?

Yeah, it definitely applies to all aspects of HBase. WALs is probably the most prevalent area
I can think of.

bq. Hard crash can happen during regular HBase operation, do we have any automation tools
in Master to address potential issues?

For WALs, I'm sure you're well aware of all of the CleanerChore logic we have surrounding
WAL archival/removal that run in the Master. For these backup tools, it's a bit different
since things are primarily being driven by the client instead of inside of HBase itself. I'm
less asking the question "why wasn't server-side driven cleanup implemented" and more trying
to ask the question "should we implement such cleanup?". I'd defer to you to say how easy
such an automated (and safe) cleanup would be inside of the Master.

If it would be too difficult (which is what my gut-reaction was), a tool/utility to summarize
these (expected) transient data (files in HDFS and hbase:backup records) would be really nice
to have. If/when we would have to debug some kind of issue WRT backups or just HDFS use by
hbase, such a tool could give us a definitive yes/no as to whether these transient files are
to blame or not.

> HBase Backup/Restore Phase 3: Merge backup images
> -------------------------------------------------
>                 Key: HBASE-14135
>                 URL: https://issues.apache.org/jira/browse/HBASE-14135
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>            Priority: Blocker
>              Labels: backup
>             Fix For: 2.0.0
>         Attachments: HBASE-14135-v3.patch, HBASE-14135-v5.patch, HBASE-14135-v6.patch,
HBASE-14135-v7.patch, HBASE-14135-v8.patch, HBASE-14135-v9.patch
> User can merge incremental backup images into single incremental backup image.
> # Merge supports only incremental images
> # Merge supports only images for the same backup destinations
> Command:
> {code}
> hbase backup merge image1,image2,..imageK
> {code}
> Example:
> {code}
> hbase backup merge backup_143126764557,backup_143126764456 
> {code}
> When operation is complete, only the most recent backup image will be kept (in above
example -  backup_143126764557) as a merged backup image, all other images will be deleted
from both: file system and backup system tables, corresponding backup manifest for the merged
backup image will be updated to remove dependencies from deleted images. Merged backup image
will contains all the data from original image and from deleted images.

This message was sent by Atlassian JIRA

View raw message