hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2116) Job.local.dir to be exposed to tasks
Date Thu, 07 Feb 2008 07:39:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12566481#action_12566481
] 

Owen O'Malley commented on HADOOP-2116:
---------------------------------------

You are right that the file cache is shared between jobs and that is the desired behavior.
(Although it is fair to ask the question of whether that is the right policy once we have
permissions. In general, probably not since it wouldn't be hard to create a file that looks
like the desired one and get access to a file that you should have access to.)

So what would you suggest for a layout?

> Job.local.dir to be exposed to tasks
> ------------------------------------
>
>                 Key: HADOOP-2116
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2116
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.14.3
>         Environment: All
>            Reporter: Milind Bhandarkar
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.17.0
>
>         Attachments: patch-2116.txt, patch-2116.txt, patch-2116.txt, patch-2116.txt
>
>
> Currently, since all task cwds are created under a jobcache directory, users that need
a job-specific shared directory for use as scratch space, create ../work. This is hacky, and
will break when HADOOP-2115 is addressed. For such jobs, hadoop mapred should expose job.local.dir
via localized configuration.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message