hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8522) Archived hfiles and old hlogs may be deleted immediatly by HFileCleaner, LogCleaner in HMaster
Date Mon, 20 May 2013 16:57:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13662146#comment-13662146
] 

Hadoop QA commented on HBASE-8522:
----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12583841/HBASE-8522-trunk-v2.patch
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 3 new or modified
tests.

    {color:green}+1 hadoop1.0{color}.  The patch compiles against the hadoop 1.0 profile.

    {color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 2.0 profile.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:red}-1 lineLengths{color}.  The patch introduces lines longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5750//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5750//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5750//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5750//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5750//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5750//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5750//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5750//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5750//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5750//console

This message is automatically generated.
                
> Archived hfiles and old hlogs may be deleted immediatly by HFileCleaner, LogCleaner in
HMaster
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-8522
>                 URL: https://issues.apache.org/jira/browse/HBASE-8522
>             Project: HBase
>          Issue Type: Bug
>    Affects Versions: 0.94.3
>            Reporter: Liu Shaohui
>            Assignee: Liu Shaohui
>            Priority: Minor
>         Attachments: HBASE-8522-0.94.patch, HBASE-8522-0.94-v2.patch, HBASE-8522-trunk.patch,
HBASE-8522-trunk-v2.patch, HBASE-8522-trunk-v2.patch
>
>
> TimeToLiveHFileCleaner is configed to 'hbase.master.hfilecleaner.plugins' in hbase-default.xml.
And timeToLiveHFileCleaner uses the modify time of the hfile to determine if it should be
deleted. But, the modify time of the hdfs file is time when its writer is closed. The rename
op will not change the modify time of the hfile. So the hfile may be deleted immediatly by
HFileCleaner after it is moved to archives. See log:
> 2013-05-08 08:15:46,053 DEBUG org.apache.hadoop.hbase.master.cleaner.CleanerChore: Checking
directory: hdfs://hbase/.archive/table/4e48ffc1ec089082c66e6d1b5f018fb5/M/729e8bc1430540cb9b2c147c90039cdc
> 2013-05-08 08:15:46,055 DEBUG org.apache.hadoop.ipc.ProtobufRpcEngine: Call: getFileInfo
took 1ms
> 2013-05-08 08:15:46,055 DEBUG org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner:
Life:40033567, ttl:300000, current:1367972146054, from: 1367932112487
> 2013-05-08 08:15:46,055 DEBUG org.apache.hadoop.hbase.master.cleaner.CleanerChore: Removing:hdfs://hbase/.archive/table/4e48ffc1ec089082c66e6d1b5f018fb5/M/729e8bc1430540cb9b2c147c90039cdc
from archive
> The same to old hlogs.
> And my solution is very simple: When hfiles and hlogs are archived, we set the modify
time of files after rename.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message