hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haohui Mai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5916) provide API to bulk delete directories/files
Date Mon, 10 Feb 2014 19:23:25 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13896889#comment-13896889

Haohui Mai commented on HDFS-5916:

I have a few questions:

# What would be the semantic of the call if one of the deletion has failed?
# Should this operation be atomic?
# When should the changes propagate to other users?
# When should happen when the operation happen in the middle of NN failover?

I can't think of good answers of any of these questions, thus it looks to me that the semantic
at the file system layer is unclear.

Maybe it is better to implement as multiple RPC calls, but the RPC the messages are sent in
the same packet. Alternatively, if you are able to put the files into a single directory then
it might solve your problem :-)

> provide API to bulk delete directories/files
> --------------------------------------------
>                 Key: HDFS-5916
>                 URL: https://issues.apache.org/jira/browse/HDFS-5916
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>            Reporter: Sergey Shelukhin
> It would be nice to have an API to delete directories and files in bulk - for example,
when deleting Hive partitions or HBase regions in large numbers, the code could avoid many
trips to NN. 

This message was sent by Atlassian JIRA

View raw message