hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Araceli Henley (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-4278) cannot run two local jobs in parallel from the same gateway.
Date Tue, 22 May 2012 19:52:41 GMT
Araceli Henley created MAPREDUCE-4278:
-----------------------------------------

             Summary: cannot run two local jobs in parallel from the same gateway.
                 Key: MAPREDUCE-4278
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4278
             Project: Hadoop Map/Reduce
          Issue Type: Bug
    Affects Versions: 1.0.2, 0.23.1, 0.20.205.0
            Reporter: Araceli Henley


I cannot run two local mode jobs from Pig in parallel from the same gateway, this is a typical
use case. If I re-run the tests sequentially, then the test pass. This seems to be a problem
from Hadoop.

Additionally, the pig harness, expects to be able to run Pig-version-undertest against Pig-version-stable
from the same gateway.


To replicate the error:

I have two clusters running from the same gateway.
If I run the Pig regression suites nightly.conf in local mode in paralell - once on each cluster.
Conflicts in M/R local mode result in failures in the tests. 


ERROR1:

org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find
output/file.out in any of the configured local directories
        at
org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:429)
        at
org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:160)
        at
org.apache.hadoop.mapred.MapOutputFile.getOutputFile(MapOutputFile.java:56)
        at org.apache.hadoop.mapred.Task.calculateOutputSize(Task.java:944)
        at org.apache.hadoop.mapred.Task.sendLastUpdate(Task.java:924)
        at org.apache.hadoop.mapred.Task.done(Task.java:875)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:374)
---

ERROR2:

2012-05-17 20:25:36,762 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
-
HadoopJobId: job_local_0001
2012-05-17 20:25:36,778 [Thread-3] INFO  org.apache.hadoop.mapred.Task -
Using ResourceCalculatorPlugin : org.apache.
hadoop.util.LinuxResourceCalculatorPlugin@ffa490e
2012-05-17 20:25:36,837 [Thread-3] WARN
org.apache.hadoop.mapred.LocalJobRunner - job_local_0001
java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
        at java.util.ArrayList.RangeCheck(ArrayList.java:547)
        at java.util.ArrayList.get(ArrayList.java:322)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getLoadFunc(PigInputFormat.java
:153)
        at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.createRecordReader(PigInputForm
at.java:106)
        at
org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.<init>(MapTask.java:489)
        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:731)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
        at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)
2012-05-17 20:25:41,291 [main] INFO
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message