hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: [PROPOSAL] introduce Python as build-time and run-time dependency for Hadoop and throughout Hadoop stack
Date Thu, 22 Nov 2012 09:02:57 GMT
On 22 November 2012 02:40, Chris Nauroth <cnauroth@hortonworks.com> wrote:

> It seems like the trickiest issue is preservation of permissions and
> symlinks in tar files.  I suspect that any JVM-based solution like custom
> Maven plugins, Groovy, or jtar would be limited in this respect.  According
> to Ant documentation, it's a JDK limitation, so I suspect all of these
> would have the same problem.  I haven't tried any of them though.  (If
> there was a feasible solution, then Ant likely would have incorporated it
> long ago.)  If anyone wants to try though, we might learn something from
> that.
> Thank you,
> --Chris
You are limited by what File.canRead(), canWrite() and canExecute) tell you.

The absence of a way to detect file permissions in Java -is because of the
lowest-common-denominator approach of the JavaFS APIs, supporting FAT32
(odd case logic, no perms or symlinks), NTFS (odd case logic, ACLs over
perms, symlinks historically very hard to create), HFS+ (case insensitive
unix fs!) as well as classic unixy filesystems.

Ant <tarfileset> filesets in <tar> let you spec permissions on filesets you
pull into the tar; they are generated x-platform, which the other reason
why you declare them in <tar> -you have the right to generate proper tar
files even if you use a Windows box.

symlinks are problematic -even detecting them cross platform is pretty
unreliable. To really do them you'd need to add a new <symlinkfileset>
entity for <tar>, that would take the link declaration. I could imagine how
to do that -and if stuck into the hadoop tools JAR, wouldn't even depend on
a new version of Ant.

Maven just adds extra layers in the way.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message