hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerry He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14391) Empty regionserver WAL will never be deleted although the coresponding regionserver has been stale
Date Fri, 18 Sep 2015 16:54:04 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14875940#comment-14875940
] 

Jerry He commented on HBASE-14391:
----------------------------------

I attached a v3 patch, which I tested it on a cluster.
The patch still does not work.
Now I understand why HBASE-12522 removed the code to delete the logDir.  Because now there
can be multiple WALProvider sharing the same logDir.
For example, there is metaProvider and a default provider on the region server that holds
the meta.  There are two FSHLog instances.
When we close the metaProvider, we only archive its log files.  We can not delete logDir because
it may still contain log files from the other instance.

> Empty regionserver WAL will never be deleted although the coresponding regionserver has
been stale
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-14391
>                 URL: https://issues.apache.org/jira/browse/HBASE-14391
>             Project: HBase
>          Issue Type: Bug
>          Components: wal
>    Affects Versions: 1.0.2
>            Reporter: Qianxi Zhang
>            Assignee: Qianxi Zhang
>         Attachments: HBASE-14391-master-v3.patch, HBASE_14391_trunk_v1.patch, HBASE_14391_trunk_v2.patch,
WALs-leftover-dir.txt
>
>
> When I restarted the hbase cluster in which there was few data, I found there are two
directories for one host with different timestamp which indicates that the old regionserver
wal directory is not deleted.
> FHLog#989
> {code}
>  @Override
>   public void close() throws IOException {
>     shutdown();
>     final FileStatus[] files = getFiles();
>     if (null != files && 0 != files.length) {
>       for (FileStatus file : files) {
>         Path p = getWALArchivePath(this.fullPathArchiveDir, file.getPath());
>         // Tell our listeners that a log is going to be archived.
>         if (!this.listeners.isEmpty()) {
>           for (WALActionsListener i : this.listeners) {
>             i.preLogArchive(file.getPath(), p);
>           }
>         }
>         if (!FSUtils.renameAndSetModifyTime(fs, file.getPath(), p)) {
>           throw new IOException("Unable to rename " + file.getPath() + " to " + p);
>         }
>         // Tell our listeners that a log was archived.
>         if (!this.listeners.isEmpty()) {
>           for (WALActionsListener i : this.listeners) {
>             i.postLogArchive(file.getPath(), p);
>           }
>         }
>       }
>       LOG.debug("Moved " + files.length + " WAL file(s) to " +
>         FSUtils.getPath(this.fullPathArchiveDir));
>     }
>     LOG.info("Closed WAL: " + toString());
>   }
> {code}
> When regionserver is stopped, the hlog will be archived, so wal/regionserver is empty
in hdfs.
> MasterFileSystem#252
> {code}
>         if (curLogFiles == null || curLogFiles.length == 0) {
>             // Empty log folder. No recovery needed
>             continue;
>           }
> {code}
> The regionserver directory will be not splitted, it makes sense. But it will be not deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message