hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13622) `-atomic` should not be supported while using `distcp` command in object file system
Date Thu, 22 Sep 2016 10:07:21 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15512833#comment-15512833

Steve Loughran commented on HADOOP-13622:

no, no timetable.

What would be good would be some special section in {{hadoop-common-project/hadoop-common/src/site/markdown/FileSystemShell.md}}
on "fs shell commands and object stores"

* why you can get not found exceptions on an {{-fs ls s3a://bucket }} URL if you leave off
the trailing /  (it's because there's no homedir, and the shell assumes its a relative path)
* distcp atomic not working
* why you should use {{fs -put -d}} to upload files without the rename
* why you should use {{-skipTrash}} when deleting things
* why you shouldn't put secrets in the URLs, e.g.  s3n://awsId:secret@mybucket/

Looks also like the document doesn't even mention put's -d and -f options, so it could do
with a quick comparison of what usage info gets printed in a {{hadoop fs}} message and what
the docs say, updating the docs.

Would you be willing to do this?

> `-atomic` should not be supported while using `distcp` command in object file system
> ------------------------------------------------------------------------------------
>                 Key: HADOOP-13622
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13622
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Yuanbo Liu
>            Assignee: Yuanbo Liu
> After discussing with [~stevel@apache.org] in HADOOP-13593, I get the point that none
of the object stores support atomic renames. So I file a new jira and ready to provide a patch
to disable `distcp -atomic`  in object file system.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message