hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-18399) Files in a snapshot can go missing even after the snapshot is taken successfully
Date Tue, 18 Jul 2017 10:33:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-18399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091390#comment-16091390
] 

ramkrishna.s.vasudevan commented on HBASE-18399:
------------------------------------------------

bq.store_file_A is marked as compacted away and HFileArchiver moves the file to archive.
Ya so the file is in archive. The remaining steps are irrespective of the store file accounting
feature right? Just asking.

> Files in a snapshot can go missing even after the snapshot is taken successfully
> --------------------------------------------------------------------------------
>
>                 Key: HBASE-18399
>                 URL: https://issues.apache.org/jira/browse/HBASE-18399
>             Project: HBase
>          Issue Type: Sub-task
>          Components: regionserver, Scanners
>            Reporter: Ashu Pachauri
>             Fix For: 1.3.2
>
>
> Files missing after the snapshot is taken (only applicable when the TTL for the TimeToLiveHFileCleaner
is small, like the default 5 mins)
>     * SnapshotManifest#addRegion visits store_file_A, but is yet to write it to the manifest.
>     * store_file_A is marked as compacted away and HFileArchiver moves the file to archive.
>     * HFileCleaner comes in and sees the store_file_A in archive. It adds the file to
the list of files that might need to be cleaned up.
>     * HFileCleaner's SnapshotHFileCleaner plugin is kicked in.
>     * SnapshotFileCache#getUnreferencedFiles also says that store_file_A is unreferenced
and should be cleaned up (It has not yet been written to the manifest).
>     * SnapshotHFileCleaner is still going through rest of the files in archive.
>     * store_file_A reference is created and written to snapshot manifest.
>     * Snapshot verification runs and sees the store_file_A is present in archive, and
thus the verification passes.
>     * Now, the SnapshotHFileCleaner finishes and TimeToLiveHFileCleaner is triggered.
If TTL has passed since the store_file_A was moved to archive (SnapshotHFileCleaner could
take easily several minutes to go through rest of the files), the TimeToLiveHFileCleaner also
marks the file as deletable.
>     * Since all cleaner plugins marked file as deletable, the store_file_A is deleted.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message