ant-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefan Bodewig <bode...@apache.org>
Subject Re: Question about using Tar with Hadoop files
Date Thu, 22 Dec 2011 14:50:05 GMT
On 2011-12-19, Frank Astier wrote:

> I’m trying to use the Apache Tar package (1.8.2) for a Java program
> that tars large files in Hadoop. I am currently failing on a file
> that’s 17 GB long.

First of all, do yourself a favor and use Commons Compress rather than
Ant's tar package.

Traditional tar doesn't support anything bigger than 2GB (an octal
number of eleven sevens).  Ant's tar package doesn't go beyond that.

Later versions of tar support workarounds (namely ustar later used by
GNU tar and BSD tar aswell) and even later the newer POSIX standard
added PAX extension headers to address this (and other things like file
names longer than 100 characters).

The trunk of Commons Compress supports both ustar and PAX by now but the
latest release of it doesn't.  I expect the next release of Commons
Compress to happen pretty soon, though.

Stefan

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@ant.apache.org
For additional commands, e-mail: user-help@ant.apache.org


Mime
View raw message