hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei-Chiu Chuang (JIRA)" <j...@apache.org>
Subject [jira] [Assigned] (HDFS-13529) Fix default trash policy emptier trigger time correctly
Date Wed, 31 Jul 2019 12:57:00 GMT

     [ https://issues.apache.org/jira/browse/HDFS-13529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Wei-Chiu Chuang reassigned HDFS-13529:
--------------------------------------

    Assignee: He Xiaoqiao

> Fix default trash policy emptier trigger time correctly
> -------------------------------------------------------
>
>                 Key: HDFS-13529
>                 URL: https://issues.apache.org/jira/browse/HDFS-13529
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 3.1.0, 2.7.6, 3.2.0, 2.9.2, 2.8.5
>            Reporter: He Xiaoqiao
>            Assignee: He Xiaoqiao
>            Priority: Major
>         Attachments: HDFS-13529-trunk.001.patch
>
>
> Trash emptier is designed to auto trigger at UTC 00:00, however I am confused all the
time that it usually triggers at a few minutes even half a hour after UTC 00:00 actually in
our production cluster.
> The main reason is default policy emptier thread sleep more time than as expect, since
it does not consider the delete operation time cost itself. especially for a large cluster,
auto trash cleaner may cost dozens of minutes.
> The right way is that gets current time {{now}} before calculate {{end}} time.
> {code:java}
>       long now = Time.now();
>       while (true) {
>         end = ceiling(now, emptierInterval);
>         try {                                     // sleep for interval
>           Thread.sleep(end - now);
>         } catch (InterruptedException e) {
>           break;                                  // exit on interrupt
>         }
>         try {
>           now = Time.now();
>           ...... // delete trash checkpoint
>         } catch (Exception e) {
>           LOG.warn("RuntimeException during Trash.Emptier.run(): ", e); 
>         }
>       }
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message