hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thejas M Nair (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-103) Shared Job /tmp location should be configurable
Date Fri, 06 Aug 2010 18:46:17 GMT

    [ https://issues.apache.org/jira/browse/PIG-103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12896116#action_12896116

Thejas M Nair commented on PIG-103:

I think using ".dir" instead of ".loc" in the property name would be better, it be consistent
with mapred.temp.dir , and also similar to java.io.tempdir. 

> Shared Job /tmp location should be configurable
> -----------------------------------------------
>                 Key: PIG-103
>                 URL: https://issues.apache.org/jira/browse/PIG-103
>             Project: Pig
>          Issue Type: Improvement
>          Components: impl
>         Environment: Partially shared file:// filesystem (eg NFS)
>            Reporter: Craig Macdonald
>            Assignee: niraj rai
>             Fix For: 0.8.0
>         Attachments: conf_tmp_dir.patch
> Hello,
> I'm investigating running pig in an environment where various parts of the file:// filesystem
are available on all nodes. I can tell hadoop to use a file:// file system location for it's
default, by seting fs.default.name=file://path/to/shared/folder
> However, this creates issues for Pig, as Pig writes it's job information in a folder
that it assumes is a shared FS (eg DFS). However, in this scenario /tmp is not shared on each
> So /tmp should either be configurable, or Hadoop should tell you the actual full location
set in fs.default.name?
> Straightforward solution is to make "/tmp/" a property in src/org/apache/pig/impl/io/FileLocalizer.java
> Any suggestions of property names?

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message