hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1961) -get, -copyToLocal fail when single filename is passed
Date Thu, 04 Oct 2007 00:36:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532303
] 

Tsz Wo (Nicholas), SZE commented on HADOOP-1961:
------------------------------------------------

+1
Patch looks good.  Below are some minor thoughts.

- I totally agree that there are redundant copyToLocal methods.  See also HADOOP-1544.

- In the case of rename failing, we know that the tmp file is good but cannot be renamed to
dst.  User can easily rename the tmp file.  For the other exception cases, user don't know
what to do to fix the problem.  If we remove "deleteOnExit" flag, then we cannot easily tell
whether the tmp file is perfect or not.

- Should we not use deprecated API anymore?  e.g. srcFS.isDirectory(src), srcFS.listPaths(src)

> -get, -copyToLocal fail when  single filename is passed
> -------------------------------------------------------
>
>                 Key: HADOOP-1961
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1961
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.14.1
>            Reporter: Koji Noguchi
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.14.2
>
>         Attachments: HADOOP-1961-branch14.patch, HADOOP-1961.patch, HADOOP-1961.patch,
HADOOP-1961.patch
>
>
> In 0.14.1 and in trunk, when I try 
> % hadoop dfs -get /user/knoguchi/aaa  aaa
> get: Failed to rename tmp file to local destination "aaa".  Remote source file "/user/knoguchi/aaa"
is saved to "/tmp/_copyToLocal_aaa30478".
> This works. 
> % hadoop dfs -get /user/knoguchi/aaa  ./aaa
> or 
> % hadoop dfs -get /user/knoguchi/aaa   /home/knoguchi/aaa
> My guess. With change in HADOOP-1292, it now creates a tmp file when -copyToLocal.
> When destination path is passed without any directory, tmp file is created under '/tmp'.
Otherwise, it uses the same directory as the destination path.
> In Java API for File.renameTo, 
> http://java.sun.com/javase/6/docs/api/java/io/File.html#renameTo(java.io.File)
> it says 
> " The rename operation might not be able to move a file from one filesystem to another",

> so renameTo call from /tmp/_tmpfile to /home/knoguchi can fail.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message