hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj K (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-1745) FileInputFormat.setInputPaths() has sideffects on the passed conf besides setting the input.dir path
Date Tue, 26 Jul 2011 17:59:10 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13071246#comment-13071246
] 

Devaraj K commented on MAPREDUCE-1745:
--------------------------------------

As per the code, when we give relative dir it is always resolving to the home dir of proxy
user and not the super user. 

I am not able to reproduce this issue. Is there any specific case to reproduce it?


> FileInputFormat.setInputPaths() has sideffects on the passed conf besides setting the
input.dir path
> ----------------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1745
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1745
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Dick King
>
> it sets other props, like {{mapred.working.dir}} , it sets this using the current user
(which may be the superuser), then
> the job submission for a proxy user fails because the {{mapred.working.dir}} is not the
right one for the proxy user.
> This is observed when using relative directories, they are resolved to the homeDir of
the superuser instead of the
> proxy user.
> I did not check, but I suspect the {{FileOutputForamt.setOutputPath()}} may have similar
side effects
> There is a workaround, setting up the {{mapred.input.dir}} and {{mapred.output.dir}}
directly by hand in the conf instead using
> the methods above

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message