hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1544) combine FsShell.copyToLocal to ChecksumFileSystem.copyToLocalFile
Date Thu, 28 Jun 2007 21:05:04 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12508931

Tsz Wo (Nicholas), SZE commented on HADOOP-1544:

- I think we mainly need a single copy method in FileSystem.  Since FileSystem.copyToLocal
and FileSystem.copyFromLocal ultimately call 
FileUtil.copy(FileSystem srcFS, Path src, FileSystem dstFS, Path dst, boolean deleteSource,
Configuration conf),
which don't care whether the file systems are local or not.

- If we want to keep FileSystem.copyToLocal and FileSystem.copyFromLocal, make them final
and calling FileSystem.copy (a new method).

- In FsShell, the command copyToLocal first copy src to a tmp file and then rename it (see
HADOOP-1292).  So, it depends on whether we want this behavior everywhere.

- One more concern is in ChecksumFileSystem:  It checks whether the CRC files should be copied.

- There are quite a few methods check whether the src or dst is a file or directory.  It is
better to make such checking in one place.

> combine FsShell.copyToLocal to ChecksumFileSystem.copyToLocalFile
> -----------------------------------------------------------------
>                 Key: HADOOP-1544
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1544
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: dfs
>            Reporter: Tsz Wo (Nicholas), SZE
>            Assignee: Tsz Wo (Nicholas), SZE
>            Priority: Minor
> - Two methods provide similar functions
> - ChecksumFileSystem.copyToLocalFile(Path src, Path dst, boolean copyCrc) is no longer
used anywhere in the system
> - It is better to use ChecksumFileSystem.getRawFileSystem() for copying crc in FsShell.copyToLocal
> - FileSystem.isDirectory(Path) used in FsShell.copyToLocal is deprecated.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message