hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2461) Hudson jobs failing because mapred staging directory is full
Date Thu, 10 May 2012 00:36:51 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13271984#comment-13271984
] 

Luke Lu commented on MAPREDUCE-2461:
------------------------------------

We ran into this issue as well. The problem is from the staging area dirs from jobs in local
mode (via LocalJobRunner), which creates a staging area dir as <staging_root>/<user><random>/.staging
instead of just <staging_root>/<user>/.staging in cluster mode. The issue is introduced
with the security releases (since 0.20.20x) when the getStageAreaDir API is introduced.

The random number for the local mode is presumably used to avoid job collisions, since there
is no jobtracker to issue unique job ids.

Mayb we can introduce a feature (mapreduce.job.staging.keep=<number of latest jobs to keep>)
to prune these directories once in a while.
                
> Hudson jobs failing because mapred staging directory is full
> ------------------------------------------------------------
>
>                 Key: MAPREDUCE-2461
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2461
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.23.0
>            Reporter: Todd Lipcon
>             Fix For: 0.24.0
>
>
> All of the tests that submit MR jobs are failing on the h7 build machine. This is because
the staging directory is entirely full:
> hudson@h7:/tmp/mr/mr$ ls -l /tmp/hadoop-hudson/mapred/staging/ | wc -l
> 31999
> This makes me think that there's some bug where we're leaking things in the staging directory.
I will manually clean this for now, but we should investigate.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message