hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-369) Added ability to copy all part-files into one output file
Date Wed, 19 Jul 2006 07:40:14 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-369?page=comments#action_12422050 ] 
            
Doug Cutting commented on HADOOP-369:
-------------------------------------

I don't see why this needs to be a new generic FileSystem method.  Can't it simply be a FileUtil
method?

Alternately, couldn't we just modify DFSShell.cat(String) to, when it's argument is a directory,
recursively descend and copy all contained file content to standard output?  Then one could
just 'bin/hadoop dfs cat foo > bar' to append all content under 'foo' into a local file
'bar'.  Would that work for you?

> Added ability to copy all part-files into one output file
> ---------------------------------------------------------
>
>                 Key: HADOOP-369
>                 URL: http://issues.apache.org/jira/browse/HADOOP-369
>             Project: Hadoop
>          Issue Type: New Feature
>          Components: dfs
>    Affects Versions: 0.4.0
>            Reporter: Johan Oskarson
>            Priority: Trivial
>         Attachments: copymerge.patch
>
>
> Since we use the hadoop output in non-hadoop applications it's nice to be able to merge
the part-files into one output file on the local filesystem.
> So I've added a dfsshell feature that streams from all files in a directory to one output
file.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message