hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaoyu Yao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12358) FSShell should prompt before deleting directories bigger than a configured size
Date Mon, 31 Aug 2015 17:48:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12358?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14723767#comment-14723767
] 

Xiaoyu Yao commented on HADOOP-12358:
-------------------------------------

Thanks all for the review and feedbacks. Update to patch v6 with the following summary of
changes based on the feedback:
1. add "-safely" option to -rm command as [~aw] suggested.
2. Reduce the configuration complexity as [~arpitagarwal] suggested. 
Only one key "hadoop.shell.delete.limit.num.files" is used.  
3. Document 'hadoop.shell.delete.limit.num.files' in core-default.xml.
4. Make this feature optional and off by default to avoid breaking any existing automations.
It is enabled only if all the three criteria are met:
* Trash is not enabled or unable to protect the directory to be deleted
* and -safely is used in the rm command
* and hadoop.shell.delete.limit.num.files > 0
This way, the admin can choose if they think the feature is useful for certain use cases.
Especially with HADOOP-11353, admin can alias 'hadoop -rm' to 'hadoop -rm -safely' in .hadooprc
like 'rm' to 'rm -i' in Linux deployments when necessary. 

[~arpitagarwal]: Given HDFS-4995 and HDFS-8046 have improved the NN locking issue of getContentSummary,
is it OK to investigate  for performance improvement in a separate JIRA?

> FSShell should prompt before deleting directories bigger than a configured size
> -------------------------------------------------------------------------------
>
>                 Key: HADOOP-12358
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12358
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>            Reporter: Xiaoyu Yao
>            Assignee: Xiaoyu Yao
>         Attachments: HADOOP-12358.00.patch, HADOOP-12358.01.patch, HADOOP-12358.02.patch,
HADOOP-12358.03.patch, HADOOP-12358.04.patch, HADOOP-12358.05.patch, HADOOP-12358.06.patch
>
>
> We have seen many cases with customers deleting data inadvertently with -skipTrash. The
FSShell should prompt user if the size of the data or the number of files being deleted is
bigger than a threshold even though -skipTrash is being used.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message