Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AF6BADF8F for ; Mon, 20 May 2013 16:38:15 +0000 (UTC) Received: (qmail 37693 invoked by uid 500); 20 May 2013 16:31:56 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 21488 invoked by uid 500); 20 May 2013 16:28:33 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 5515 invoked by uid 99); 20 May 2013 15:33:17 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 20 May 2013 15:33:17 +0000 Date: Mon, 20 May 2013 15:33:17 +0000 (UTC) From: "Matteo Bertozzi (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-8522) Archived hfiles and old hlogs may be deleted immediatly by HFileCleaner, LogCleaner in HMaster MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13662083#comment-13662083 ] Matteo Bertozzi commented on HBASE-8522: ---------------------------------------- Hey [~lhofhansl] does my answer make sense to you? are you ok with the patches? > Archived hfiles and old hlogs may be deleted immediatly by HFileCleaner, LogCleaner in HMaster > ---------------------------------------------------------------------------------------------- > > Key: HBASE-8522 > URL: https://issues.apache.org/jira/browse/HBASE-8522 > Project: HBase > Issue Type: Bug > Affects Versions: 0.94.3 > Reporter: Liu Shaohui > Assignee: Liu Shaohui > Priority: Minor > Attachments: HBASE-8522-0.94.patch, HBASE-8522-0.94-v2.patch, HBASE-8522-trunk.patch, HBASE-8522-trunk-v2.patch > > > TimeToLiveHFileCleaner is configed to 'hbase.master.hfilecleaner.plugins' in hbase-default.xml. And timeToLiveHFileCleaner uses the modify time of the hfile to determine if it should be deleted. But, the modify time of the hdfs file is time when its writer is closed. The rename op will not change the modify time of the hfile. So the hfile may be deleted immediatly by HFileCleaner after it is moved to archives. See log: > 2013-05-08 08:15:46,053 DEBUG org.apache.hadoop.hbase.master.cleaner.CleanerChore: Checking directory: hdfs://hbase/.archive/table/4e48ffc1ec089082c66e6d1b5f018fb5/M/729e8bc1430540cb9b2c147c90039cdc > 2013-05-08 08:15:46,055 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: getFileInfo took 1ms > 2013-05-08 08:15:46,055 DEBUG org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner: Life:40033567, ttl:300000, current:1367972146054, from: 1367932112487 > 2013-05-08 08:15:46,055 DEBUG org.apache.hadoop.hbase.master.cleaner.CleanerChore: Removing:hdfs://hbase/.archive/table/4e48ffc1ec089082c66e6d1b5f018fb5/M/729e8bc1430540cb9b2c147c90039cdc from archive > The same to old hlogs. > And my solution is very simple: When hfiles and hlogs are archived, we set the modify time of files after rename. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira