hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3103) [HOD] Hadoop.tmp.dir should not be set to cluster directory
Date Thu, 27 Mar 2008 09:55:24 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vinod Kumar Vavilapalli updated HADOOP-3103:
--------------------------------------------

    Attachment: HADOOP-3103.for.17
                HADOOP-3103.for.16

Attaching two patches: HADOOP-3103.for.16 and HADOOP-3103.for.17.
   * The patch for Hadoop 17 has testcases, while the one for Hadoop 16 doesn't (and it cannot
have, 'coz the testcase framework is not there in 16).
   * testcases are written for verifying the presence of the parameter only on the client
side. Testcases for server side are more involving. Skipping them for now, seems ok, 'coz,
as of now, the parameter is not going to be used at all on the server side.
   * As per Hemanth's comments(outside Jira), making hadoop.tmp.dir as a single value parameter
instead of a comma separated list, which seems to be how hadoop.tmp.dir is being used currently.
   * Hadoop.tmp.dir *is* marked final on daemons side, so client side config parameter won't
override this on the server side.

> [HOD] Hadoop.tmp.dir should not be set to cluster directory
> -----------------------------------------------------------
>
>                 Key: HADOOP-3103
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3103
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/hod
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Vinod Kumar Vavilapalli
>            Priority: Blocker
>             Fix For: 0.16.2
>
>         Attachments: HADOOP-3103, HADOOP-3103.for.16, HADOOP-3103.for.17
>
>
> Currently HOD generates  hadoop.tmp.dir pointing to the cluster directory(hod.clusterdir),
which users typically create in their home directories. This hadoop.tmp.dir is used on the
client side when a jar is run(org/apache/hadoop/util/RunJar.java). The user home directories
might be NFS mounted in some environments, which might result in a hit, for example when running
a jar.
> So, HOD *should not* generate hadoop.tmp.dir pointing to cluster directory. Instead,
it should use hod.temp-dir, which *is* the location that dfs.client.buffer.dir uses to alleviate
any such NFS hit problems. Further, though currently hadoop.tmp.dir is not used on hadoop
daemons side, it would be good if HOD generates it on daemons side too, and thus preclude
any related problems in future.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message