hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4964) JobLocalizer#localizeJobFiles can potentially write job.xml to the wrong user's directory
Date Fri, 01 Feb 2013 22:26:13 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13569168#comment-13569168

Karthik Kambatla commented on MAPREDUCE-4964:

Thanks Arun - the other patch completely skipped my mind. I agree that the patch on MAPREDUCE-4843
would fix the immediate issue at hand - jobconf being written to a different user's directory.
However, this patch fixes a fundamental issue in {{JobLocalizer}} that the test in the patch
clearly exposes.

Let me explain through the code (with comments added after each line, irrelevant lines removed).
The code before the patch:

  public JobLocalizer(JobConf ttConf, String user, String jobid,
      String... localDirs) throws IOException {
    this.ttConf = ttConf;                        // shallow copy without any indication that
ttConf can't be changed later, no obvious locks held either
    lfs = FileSystem.getLocal(ttConf).getRaw();  // ttConf could have changed already, and
we could potentially read wrong filesystem - agree highly unlikely
    ttConf.setStrings(JOB_LOCAL_CTXT, localDirs);// setting the dirs on the external ttConf,
not the instance specific one

I don't think JobLocalizer should be making these assumptions (irrespective of how it is accessed),
and if there is a strong enough reason to make these, we should communicate it to the users
of JobLocalizer.
> JobLocalizer#localizeJobFiles can potentially write job.xml to the wrong user's directory
> -----------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-4964
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4964
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv1
>    Affects Versions: 1.1.1
>            Reporter: Karthik Kambatla
>            Assignee: Karthik Kambatla
>         Attachments: MR-4964.patch, MR-4964.patch
> In the following code, if jobs corresponding to different users (X and Y) are localized
simultaneously, it is possible that jobconf can be written to the wrong user's directory.
(X's job.xml can be written to Y's directory)
> {code}
>   public void localizeJobFiles(JobID jobid, JobConf jConf,
>       Path localJobTokenFile, TaskUmbilicalProtocol taskTracker)
>       throws IOException, InterruptedException {
>     localizeJobFiles(jobid, jConf,
>         lDirAlloc.getLocalPathForWrite(JOBCONF, ttConf), localJobTokenFile,
>         taskTracker);
>   }
> {code}

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message