pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Dai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (PIG-3482) Mapper only Jobs are not creating intermediate files in /tmp/, instead of creating in user directory.
Date Sat, 11 Oct 2014 07:22:33 GMT

    [ https://issues.apache.org/jira/browse/PIG-3482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14168041#comment-14168041
] 

Daniel Dai commented on PIG-3482:
---------------------------------

[~ravitejach]Which intermediate outputs are you talking about? A map only job does not generate
any intermediate output.

> Mapper only Jobs are not creating intermediate files in /tmp/, instead of creating in
user directory. 
> ------------------------------------------------------------------------------------------------------
>
>                 Key: PIG-3482
>                 URL: https://issues.apache.org/jira/browse/PIG-3482
>             Project: Pig
>          Issue Type: Bug
>          Components: impl
>    Affects Versions: 0.11.1
>         Environment: RHEL 6.0
>            Reporter: Raviteja Chirala
>            Priority: Minor
>             Fix For: 0.14.0
>
>
> When we run Mapper only jobs, All the intermediate outputs(compressed) are going to the
user directory instead of going to tmp. If we run on small datasets, it shouldn't create a
problem. But when I run for large datasets like more than 100TB lets say, it taking up so
much disk space exceeding the disk space quota(setSpaceQuota) of 100GB also. Problem is happening
before clean up. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message