Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 18 Sep 2015 16:54:04 +0000 (UTC)
From: "Jerry He (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: <JIRA.12862692.1441786368000.13726.1442595244388@Atlassian.JIRA>
In-Reply-To: <JIRA.12862692.1441786368000@Atlassian.JIRA>
References: <JIRA.12862692.1441786368000@Atlassian.JIRA>
 <JIRA.12862692.1441786368247@arcas>
Subject: [jira] [Commented] (HBASE-14391) Empty regionserver WAL will never
 be deleted although the coresponding regionserver has been stale
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-14391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14875940#comment-14875940 ] 

Jerry He commented on HBASE-14391:
----------------------------------

I attached a v3 patch, which I tested it on a cluster.
The patch still does not work.
Now I understand why HBASE-12522 removed the code to delete the logDir.  Because now there can be multiple WALProvider sharing the same logDir.
For example, there is metaProvider and a default provider on the region server that holds the meta.  There are two FSHLog instances.
When we close the metaProvider, we only archive its log files.  We can not delete logDir because it may still contain log files from the other instance.

> Empty regionserver WAL will never be deleted although the coresponding regionserver has been stale
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-14391
>                 URL: https://issues.apache.org/jira/browse/HBASE-14391
>             Project: HBase
>          Issue Type: Bug
>          Components: wal
>    Affects Versions: 1.0.2
>            Reporter: Qianxi Zhang
>            Assignee: Qianxi Zhang
>         Attachments: HBASE-14391-master-v3.patch, HBASE_14391_trunk_v1.patch, HBASE_14391_trunk_v2.patch, WALs-leftover-dir.txt
>
>
> When I restarted the hbase cluster in which there was few data, I found there are two directories for one host with different timestamp which indicates that the old regionserver wal directory is not deleted.
> FHLog#989
> {code}
>  @Override
>   public void close() throws IOException {
>     shutdown();
>     final FileStatus[] files = getFiles();
>     if (null != files && 0 != files.length) {
>       for (FileStatus file : files) {
>         Path p = getWALArchivePath(this.fullPathArchiveDir, file.getPath());
>         // Tell our listeners that a log is going to be archived.
>         if (!this.listeners.isEmpty()) {
>           for (WALActionsListener i : this.listeners) {
>             i.preLogArchive(file.getPath(), p);
>           }
>         }
>         if (!FSUtils.renameAndSetModifyTime(fs, file.getPath(), p)) {
>           throw new IOException("Unable to rename " + file.getPath() + " to " + p);
>         }
>         // Tell our listeners that a log was archived.
>         if (!this.listeners.isEmpty()) {
>           for (WALActionsListener i : this.listeners) {
>             i.postLogArchive(file.getPath(), p);
>           }
>         }
>       }
>       LOG.debug("Moved " + files.length + " WAL file(s) to " +
>         FSUtils.getPath(this.fullPathArchiveDir));
>     }
>     LOG.info("Closed WAL: " + toString());
>   }
> {code}
> When regionserver is stopped, the hlog will be archived, so wal/regionserver is empty in hdfs.
> MasterFileSystem#252
> {code}
>         if (curLogFiles == null || curLogFiles.length == 0) {
>             // Empty log folder. No recovery needed
>             continue;
>           }
> {code}
> The regionserver directory will be not splitted, it makes sense. But it will be not deleted.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)