hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rémy SAISSY (JIRA) <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-5264) FileAlreadyExistsException is assumed to be thrown by FileSystem#mkdirs or FileContext#mkdir in the codebase
Date Fri, 07 Jun 2013 21:48:20 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-5264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Rémy SAISSY updated MAPREDUCE-5264:

    Target Version/s: 3.0.0, 2.1.0-beta
              Status: Patch Available  (was: Open)

in JobHistoryEverntHandler and HistoryFileManager, the FileAlreadyExistsException was only
tracked for Logging purpose.

YarnChild.java::configureLocalDirs() tracked the FileAlreadyExistsException to determine which
task should open the directory for write and which one for read.

      workDir = lDirAlloc.getLocalPathForWrite("work", job);
      FileSystem lfs = FileSystem.getLocal(job).getRaw();
      boolean madeDir = false;
      try {
        madeDir = lfs.mkdirs(workDir);
      } catch (FileAlreadyExistsException e) {
        madeDir = true;
        workDir = lDirAlloc.getLocalPathToRead("work", job);

However the method's goal according to its documentation is to create the mapred-local output
directory. Shouldn't it be writable anyway?
I have removed the try catch and the read only open in the patch.

No need to modify or add any test case.
> FileAlreadyExistsException is assumed to be thrown by FileSystem#mkdirs or FileContext#mkdir
in the codebase
> ------------------------------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-5264
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5264
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 3.0.0, 2.1.0-beta
>            Reporter: Rémy SAISSY
> According to https://issues.apache.org/jira/browse/HADOOP-9438,
> FileSystem#mkdirs and FileContext#mkdir do not throw FileAlreadyExistsException if the
directory already exist.
> Some places in the mapreduce codebase assumes FileSystem#mkdirs or FileContext#mkdir
throw FileAlreadyExistsException.
> At least the following files are concerned:
>  - YarnChild.java
>  - JobHistoryEverntHandler.java
>  - HistoryFileManager.java
> It would be good to re-review and patch this if needed.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message