hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Raghu Angadi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1961) -get, -copyToLocal fail when single filename is passed
Date Thu, 04 Oct 2007 00:50:50 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12532308
] 

Raghu Angadi commented on HADOOP-1961:
--------------------------------------

>     *  Should we not use deprecated API anymore? e.g. srcFS.isDirectory(src), srcFS.listPaths(src)

yeah, I saw that. I just used what ever was there before. I think it will be removed when
all the references to it are removed.

> In the case of rename failing,

With this fix rename should rarely fail. I think users will be wary of any error. The main
concern that Dhruba mentioned is while users are copying very large file and they might might
be ok with partial files. Retaining the file only in case of rename failure does not fix that
problem.


> -get, -copyToLocal fail when  single filename is passed
> -------------------------------------------------------
>
>                 Key: HADOOP-1961
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1961
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.14.1
>            Reporter: Koji Noguchi
>            Assignee: Raghu Angadi
>            Priority: Blocker
>             Fix For: 0.14.2
>
>         Attachments: HADOOP-1961-branch14.patch, HADOOP-1961.patch, HADOOP-1961.patch,
HADOOP-1961.patch
>
>
> In 0.14.1 and in trunk, when I try 
> % hadoop dfs -get /user/knoguchi/aaa  aaa
> get: Failed to rename tmp file to local destination "aaa".  Remote source file "/user/knoguchi/aaa"
is saved to "/tmp/_copyToLocal_aaa30478".
> This works. 
> % hadoop dfs -get /user/knoguchi/aaa  ./aaa
> or 
> % hadoop dfs -get /user/knoguchi/aaa   /home/knoguchi/aaa
> My guess. With change in HADOOP-1292, it now creates a tmp file when -copyToLocal.
> When destination path is passed without any directory, tmp file is created under '/tmp'.
Otherwise, it uses the same directory as the destination path.
> In Java API for File.renameTo, 
> http://java.sun.com/javase/6/docs/api/java/io/File.html#renameTo(java.io.File)
> it says 
> " The rename operation might not be able to move a file from one filesystem to another",

> so renameTo call from /tmp/_tmpfile to /home/knoguchi can fail.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message