hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Vyas <jayunit...@gmail.com>
Subject Re: JobSubmissionFiles: past , present, and future?
Date Fri, 12 Apr 2013 12:21:27 GMT
To update on this, it was just pointed out to me by matt farrallee 
that the auto fix of permissions is for a failsafe 
in case of a race condition, and not meant to mend bad permissions in all cases:

https://github.com/apache/hadoop-common/commit/f25dc04795a0e9836e3f237c802bfc1fe8a243ad

Something to keep in mind - if you see the "fixing staging permissions" error message alot
Then there might be a more systemic problem in your fs... At least, that was the case for
us.

On Apr 12, 2013, at 6:11 AM, Jay Vyas <jayunit100@gmail.com> wrote:

> Hi guys: 
> 
> I'm curious about the changes and future of the JobSubmissionFiles class.
> 
> Grepping around on the web I'm finding some code snippets that suggest that hadoop security
is not handled the same way on the staging directory as before:
> 
> http://javasourcecode.org/html/open-source/hadoop/hadoop-0.20.203.0/org/apache/hadoop/mapreduce/JobSubmissionFiles.java.html
> 
> http://mail-archives.apache.org/mod_mbox/hadoop-hdfs-user/201210.mbox/%3CCAOcnVr0EYLSckXaOCPNm7kBZWpHvCdjBBX5A+AzeS_S6PwsbkQ@mail.gmail.com%3E
> 
> But I'm having trouble definitively pinning this to versions.
> 
> Why the difference in the if/else logic here and what is the future
> Of permissions on .staging?

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message