hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J Chouraria (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-7141) hadoop fs -put and -copyFromLocal do not support globs in the source path
Date Tue, 15 Feb 2011 17:37:01 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12994893#comment-12994893

Harsh J Chouraria commented on HADOOP-7141:

@Niels - This bug probably only focuses on -put, -copyFromLocal, -moveFromLocal (too). These
functions take a list of direct, local paths and do not attempt to run a glob matcher on any.

The rm you mention should work fine in two cases:

1. Your shell tries to expand the '*', but fails to find a local path such as that, so leaves
the string intact - which goes on to be expanded by the FsShell methods.
2. Your shell finds an expandable match set, which reflect the set on the DFS also, and thus,
as both paths match, it works. (i.e. you haven't deleted the local path files and the path
on the DFS matches the exact same thing, which is the case in your example)

For example, see this case:
[QDuo hadoop-0.20.2]$ touch aa ab ac ad
[QDuo hadoop-0.20.2]$ hadoop dfs -put aa ab ac ad .
[QDuo hadoop-0.20.2]$ rm ab
[QDuo hadoop-0.20.2]$ # This should match locally, but apply to DFS!
[QDuo hadoop-0.20.2]$ hadoop dfs -rm a* 
Deleted hdfs://localhost/user/test/aa
Deleted hdfs://localhost/user/test/ac
Deleted hdfs://localhost/user/test/ad
[QDuo hadoop-0.20.2]$ hadoop dfs -ls
Found 1 items
-rw-r--r--   1 test supergroup          0 2011-02-15 22:54 /user/test/ab

> hadoop fs -put and -copyFromLocal do not support globs in the source path
> -------------------------------------------------------------------------
>                 Key: HADOOP-7141
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7141
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: fs
>    Affects Versions: 0.20.2
>         Environment: Cloudera CDH3b3
>            Reporter: Jay Hacker
>            Priority: Minor
> I'd like to be able to use Hadoop globbing with the FsShell -put command, but it doesn't
> {noformat}
> $ ls
> file1 file2
> $ hadoop fs -put '*' .
> put: File * does not exist.
> {noformat}
> This has probably gone unnoticed because your shell usually handles it, but a) I'd like
to be able to call 'hadoop fs' programatically without a shell, b) it doesn't work in Pig
or Grunt, where there is no shell helping you, and c) Hadoop globbing differs from shell globbing
and it would be nice to be able to use Hadoop globbing consistently throughout Hadoop.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message