Return-Path: X-Original-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 4AAB4114E6 for ; Wed, 30 Jul 2014 23:59:40 +0000 (UTC) Received: (qmail 37183 invoked by uid 500); 30 Jul 2014 23:59:39 -0000 Delivered-To: apmail-hadoop-mapreduce-issues-archive@hadoop.apache.org Received: (qmail 37116 invoked by uid 500); 30 Jul 2014 23:59:39 -0000 Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: mapreduce-issues@hadoop.apache.org Delivered-To: mailing list mapreduce-issues@hadoop.apache.org Received: (qmail 37054 invoked by uid 99); 30 Jul 2014 23:59:39 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 30 Jul 2014 23:59:39 +0000 Date: Wed, 30 Jul 2014 23:59:39 +0000 (UTC) From: "Allen Wittenauer (JIRA)" To: mapreduce-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Resolved] (MAPREDUCE-2072) Allow multiple instances of same local job to run simutaneously on the same machine MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/MAPREDUCE-2072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Allen Wittenauer resolved MAPREDUCE-2072. ----------------------------------------- Resolution: Fixed Stale. Probably. > Allow multiple instances of same local job to run simutaneously on the same machine > ----------------------------------------------------------------------------------- > > Key: MAPREDUCE-2072 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2072 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: job submission > Affects Versions: 0.20.2 > Reporter: Ted Yu > > On the same (build) machine, there may be multiple instances of same local job running - e.g. same unit test from snapshot build and release build. > For each build project on our build machine, there is environment variable with unique value defined. > In JobClient.submitJobInternal(), there is following code: > JobID jobId = jobSubmitClient.getNewJobId(); > Path submitJobDir = new Path(getSystemDir(), jobId.toString()); > The above code doesn't handle the scenario described previously and often leads to the following failure: > Caused by: org.apache.hadoop.util.Shell$ExitCodeException: chmod: cannot access `/tmp/hadoop-build/mapred/system/job_local_0002': No such file or directory > at org.apache.hadoop.util.Shell.runCommand(Shell.java:195) > at org.apache.hadoop.util.Shell.run(Shell.java:134) > at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:286) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:354) > at org.apache.hadoop.util.Shell.execCommand(Shell.java:337) > at org.apache.hadoop.fs.RawLocalFileSystem.execCommand(RawLocalFileSystem.java:492) > at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:484) > at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:286) > at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:308) > at org.apache.hadoop.mapred.JobClient.configureCommandLineOptions(JobClient.java:614) > at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:802) > at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:771) > at org.apache.hadoop.mapred.HadoopClient.runJob(HadoopClient.java:177) > One solution would be to incorporate the value of the underlying environment variable into either NewJobId or SystemDir so that there is no conflict. -- This message was sent by Atlassian JIRA (v6.2#6252)