hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14391) Empty regionserver WAL will never be deleted although the coresponding regionserver has been stale
Date Sun, 13 Sep 2015 13:50:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14742493#comment-14742493
] 

Hadoop QA commented on HBASE-14391:
-----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12755304/HBASE_14391_trunk_v2.patch
  against master branch at commit a9157379a3046a9d4c377675c932257c4ecb6597.
  ATTACHMENT ID: 12755304

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:red}-1 tests included{color}.  The patch doesn't appear to include any new or modified
tests.
                        Please justify why no new tests are needed for this patch.
                        Also please list what manual steps were performed to verify this patch.

    {color:green}+1 hadoop versions{color}. The patch compiles with all supported hadoop versions
(2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.0 2.7.0 2.7.1)

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 protoc{color}.  The applied patch does not increase the total number of
protoc compiler warnings.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:green}+1 checkstyle{color}.  The applied patch does not increase the total number
of checkstyle errors

    {color:green}+1 findbugs{color}.  The patch does not introduce any  new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

  {color:green}+1 site{color}.  The mvn post-site goal succeeds with this patch.

     {color:red}-1 core tests{color}.  The patch failed these unit tests:
                       org.apache.hadoop.hbase.regionserver.TestKeepDeletes
                  org.apache.hadoop.hbase.client.TestIntraRowPagination
                  org.apache.hadoop.hbase.filter.TestColumnPrefixFilter
                  org.apache.hadoop.hbase.filter.TestMultipleColumnPrefixFilter
                  org.apache.hadoop.hbase.regionserver.TestColumnSeeking
                  org.apache.hadoop.hbase.regionserver.TestHRegionInfo
                  org.apache.hadoop.hbase.filter.TestInvocationRecordFilter
                  org.apache.hadoop.hbase.io.encoding.TestPrefixTree
                  org.apache.hadoop.hbase.io.encoding.TestSeekBeforeWithReverseScan
                  org.apache.hadoop.hbase.coprocessor.TestRegionObserverStacking
                  org.apache.hadoop.hbase.regionserver.TestRegionMergeTransaction
                  org.apache.hadoop.hbase.regionserver.TestMinVersions
                  org.apache.hadoop.hbase.regionserver.TestScanner
                  org.apache.hadoop.hbase.regionserver.TestDefaultCompactSelection
                  org.apache.hadoop.hbase.io.hfile.TestScannerSelectionUsingKeyRange
                  org.apache.hadoop.hbase.regionserver.wal.TestLogRollingNoCluster
                  org.apache.hadoop.hbase.regionserver.wal.TestWALActionsListener
                  org.apache.hadoop.hbase.regionserver.TestSplitTransaction
                  org.apache.hadoop.hbase.filter.TestDependentColumnFilter
                  org.apache.hadoop.hbase.filter.TestFilter
                  org.apache.hadoop.hbase.regionserver.TestResettingCounters
                  org.apache.hadoop.hbase.coprocessor.TestCoprocessorInterface
                  org.apache.hadoop.hbase.regionserver.TestWideScanner
                  org.apache.hadoop.hbase.regionserver.TestScanWithBloomError

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/15576//testReport/
Release Findbugs (version 2.0.3) 	warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/15576//artifact/patchprocess/newFindbugsWarnings.html
Checkstyle Errors: https://builds.apache.org/job/PreCommit-HBASE-Build/15576//artifact/patchprocess/checkstyle-aggregate.html

  Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/15576//console

This message is automatically generated.

> Empty regionserver WAL will never be deleted although the coresponding regionserver has
been stale
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-14391
>                 URL: https://issues.apache.org/jira/browse/HBASE-14391
>             Project: HBase
>          Issue Type: Bug
>          Components: wal
>    Affects Versions: 1.0.2
>            Reporter: Qianxi Zhang
>            Assignee: Qianxi Zhang
>         Attachments: HBASE_14391_trunk_v1.patch, HBASE_14391_trunk_v2.patch, WALs-leftover-dir.txt
>
>
> When I restarted the hbase cluster in which there was few data, I found there are two
directories for one host with different timestamp which indicates that the old regionserver
wal directory is not deleted.
> FHLog#989
> {code}
>  @Override
>   public void close() throws IOException {
>     shutdown();
>     final FileStatus[] files = getFiles();
>     if (null != files && 0 != files.length) {
>       for (FileStatus file : files) {
>         Path p = getWALArchivePath(this.fullPathArchiveDir, file.getPath());
>         // Tell our listeners that a log is going to be archived.
>         if (!this.listeners.isEmpty()) {
>           for (WALActionsListener i : this.listeners) {
>             i.preLogArchive(file.getPath(), p);
>           }
>         }
>         if (!FSUtils.renameAndSetModifyTime(fs, file.getPath(), p)) {
>           throw new IOException("Unable to rename " + file.getPath() + " to " + p);
>         }
>         // Tell our listeners that a log was archived.
>         if (!this.listeners.isEmpty()) {
>           for (WALActionsListener i : this.listeners) {
>             i.postLogArchive(file.getPath(), p);
>           }
>         }
>       }
>       LOG.debug("Moved " + files.length + " WAL file(s) to " +
>         FSUtils.getPath(this.fullPathArchiveDir));
>     }
>     LOG.info("Closed WAL: " + toString());
>   }
> {code}
> When regionserver is stopped, the hlog will be archived, so wal/regionserver is empty
in hdfs.
> MasterFileSystem#252
> {code}
>         if (curLogFiles == null || curLogFiles.length == 0) {
>             // Empty log folder. No recovery needed
>             continue;
>           }
> {code}
> The regionserver directory will be not splitted, it makes sense. But it will be not deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message