hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Andrews <mandr...@liveops.com>
Subject Re: How to concatenate hadoop files to a single hadoop file
Date Thu, 02 Oct 2008 21:54:58 GMT

You might be able to use hars:

http://hadoop.apache.org/core/docs/current/hadoop_archives.html

On 10/2/08 2:51 PM, "Steve Gao" <steve.gao@yahoo.com> wrote:

Anybody knows? Thanks a lot.

--- On Thu, 10/2/08, Steve Gao <steve.gao@yahoo.com> wrote:
From: Steve Gao <steve.gao@yahoo.com>
Subject: How to concatenate hadoop files to a single hadoop file
To: core-user@hadoop.apache.org
Cc: core-dev@hadoop.apache.org
Date: Thursday, October 2, 2008, 3:17 PM

Suppose I have 3 files in Hadoop that I want to "cat" them to a single
file. I know it can be done by "hadoop dfs -cat" to a local file and
updating it to Hadoop. But it's very expensive for large files. Is there an
internal way to do this in Hadoop itself? Thanks








Mime
View raw message