hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amir Youssefi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (PIG-138) Support for Multiple tmp directories and clean up
Date Fri, 07 Mar 2008 00:36:58 GMT

    [ https://issues.apache.org/jira/browse/PIG-138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12575983#action_12575983
] 

Amir Youssefi commented on PIG-138:
-----------------------------------

Pi, 

 Please attach your patches here (this way you would be asked about Apache License again).

Thanks
Amir

> Support for Multiple tmp directories and clean up
> -------------------------------------------------
>
>                 Key: PIG-138
>                 URL: https://issues.apache.org/jira/browse/PIG-138
>             Project: Pig
>          Issue Type: Improvement
>         Environment: Pig Local
>            Reporter: Amir Youssefi
>
> This is to separate additional improvements and original requirement on PIG-129 issue.
> Pasting comments of Pi Song. 
> "I think the concept of multi-dir temp file creator (LocalDirAllocator in Hadoop) should
be adopted to Pig. What it does is:-
>     * You can set up a set of tmp file dirs in configuration (They can be on different
physical drives so you can utilize more disk space)
>     * When a temp file is being created, the system will probe the given temp dirs in
round-robin fashion
>     * For a selected temp dir, if it exists and you have permission to write, temp file
will be created
>     * For a selected temp dir, it it doesn't exist or you don't have permission to write,
the temp dir will be kept in the black list, thus not being used later on.
>     * For the next temp file, move on to the next temp dir
> "

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message