hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-2735) Setting default tmp directory for java createTempFile (java.io.tmpdir)
Date Fri, 08 Feb 2008 00:13:08 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-2735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Owen O'Malley updated HADOOP-2735:
----------------------------------

    Status: Open  (was: Patch Available)

-1

I think the task should fail if the create fails.

As a style point, it is better to just do
{code}
  if (a != null && a.meth()) { ... }
{code}

rather than

{code}
   if (a != null) {
      if (a.meth() { ... }
   }
{code}

> Setting default tmp directory for java createTempFile (java.io.tmpdir)
> ----------------------------------------------------------------------
>
>                 Key: HADOOP-2735
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2735
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Koji Noguchi
>            Assignee: Amareshwari Sri Ramadasu
>            Priority: Critical
>             Fix For: 0.16.1
>
>         Attachments: patch-2735.txt, patch-2735.txt
>
>
> On our cluster, we've seen Pig(http://incubator.apache.org/pig/) filling up the /tmp
and failing. 
> (also inefficient since all the local tasks were spilling to the  same disk)
> Pig is simply using java api createTempFile, 
> http://java.sun.com/j2se/1.5.0/docs/api/java/io/File.html#createTempFile(java.lang.String,%20java.lang.String,%20java.io.File
> Can we add -Djava.io.tmpdir="./tmp" somewhere ?
> so that, 
> 1) Tasks can utilize all disks when using tmp
> 2) Any undeleted tmp files will be deleted by the tasktracker when task(job?) is done.
> The easiest way is to set it inside mapred.child.java.opts in the config, but this can
be overwritten if the users set their own task heapsize.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message