hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Krishna Ramachandran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1744) DistributedCache creates its own FileSytem instance when adding a file/archive to the path
Date Fri, 21 May 2010 18:48:16 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870104#action_12870104
] 

Krishna Ramachandran commented on MAPREDUCE-1744:
-------------------------------------------------

Dick am not sure if FileSystem.get(conf) is the right call in Job.java
this returns configured file system (from doc) 

cluster.getFilesystem() gets the FileSystem where job specific files are stored

Am checking further

> DistributedCache creates its own FileSytem instance when adding a file/archive to the
path
> ------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1744
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1744
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Dick King
>         Attachments: BZ-3503564--2010-05-06.patch, h1744.patch, MAPREDUCE-1744.patch
>
>
> According to the contract of {{UserGroupInformation.doAs()}} the only required operations
within the {{doAs()}} block are the
> creation of a {{JobClient}} or getting a {{FileSystem}} .
> The {{DistributedCache.add(File/Archive)ToClasspath()}} methods create a {{FileSystem}}
instance outside of the {{doAs()}} block,
> this {{FileSystem}} instance is not in the scope of the proxy user but of the superuser
and permissions may make the method
> fail.
> One option is to overload the methods above to receive a filesystem.
> Another option is to do obtain the {{FileSystem}} within a {{doAs()}} block, for this
it would be required to have the proxy
> user set in the passed configuration.
> The second option seems nicer, but I don't know if the proxy user is as a property in
the jobconf.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message