hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-3183) Unit test fails on Windows: TestJobShell.testJobShell
Date Wed, 09 Apr 2008 00:53:24 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-3183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Douglas updated HADOOP-3183:
----------------------------------

      Resolution: Fixed
    Hadoop Flags: [Reviewed]
          Status: Resolved  (was: Patch Available)

I just committed this. Thanks, Mahadev

> Unit test fails on Windows: TestJobShell.testJobShell
> -----------------------------------------------------
>
>                 Key: HADOOP-3183
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3183
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.17.0
>            Reporter: Mukund Madhugiri
>            Assignee: Mahadev konar
>            Priority: Blocker
>             Fix For: 0.17.0
>
>         Attachments: HADOOP-3183_1.patch
>
>
> Unit test fails on Windows: TestJobShell.testJobShell
> junit.framework.AssertionFailedError: not failed 
> 	at org.apache.hadoop.mapred.TestJobShell.testJobShell(TestJobShell.java:74)
> The MR job is failing:
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,795 WARN  conf.Configuration (Configuration.java:loadResource(893))
- C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000001_0/job.xml:a
attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:35,858 INFO  jvm.JvmMetrics (JvmMetrics.java:init(67))
- Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:36,904 INFO  mapred.MapTask (MapTask.java:run(182))
- numReduceTasks: 1
> task_200804021954_0001_m_000001_0: 2008-04-02 19:54:40,356 INFO  mapred.TaskRunner (Task.java:done(387))
- Task 'task_200804021954_0001_m_000001_0' done.
> 2008-04-02 19:54:41,730 INFO  mapred.TaskInProgress (TaskInProgress.java:updateStatus(420))
- Error from task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not
found
> 	at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> 	at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,104 INFO  mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_0
done; removing files.
> 2008-04-02 19:54:45,244 INFO  mapred.JobTracker (JobTracker.java:removeMarkedTasks(974))
- Removed completed task 'task_200804021954_0001_m_000000_0' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2085'
> 2008-04-02 19:54:45,385 INFO  mapred.TaskRunner (Task.java:discardTaskOutput(502)) -
Discarded output of task 'task_200804021954_0001_m_000000_0' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_0
> 2008-04-02 19:54:45,556 INFO  mapred.JobClient (JobClient.java:runJob(1046)) - Task Id
: task_200804021954_0001_m_000000_0, Status : FAILED
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,060 WARN  conf.Configuration (Configuration.java:loadResource(893))
- C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/0_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_0/job.xml:a
attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:38,185 INFO  jvm.JvmMetrics (JvmMetrics.java:init(67))
- Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:39,028 INFO  mapred.MapTask (MapTask.java:run(182))
- numReduceTasks: 1
> task_200804021954_0001_m_000000_0: 2008-04-02 19:54:40,153 WARN  mapred.TaskTracker (TaskTracker.java:main(2118))
- Error running child
> task_200804021954_0001_m_000000_0: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_0: 	at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_0: 	at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_0: 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_0: 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_0: 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:45,963 INFO  mapred.JobInProgress (JobInProgress.java:findNewMapTask(1087))
- Choosing rack-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:45,963 INFO  mapred.JobTracker (JobTracker.java:createTaskEntry(879))
- Adding task 'task_200804021954_0001_m_000000_1' to tip tip_200804021954_0001_m_000000, for
tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:45,978 INFO  mapred.TaskTracker (TaskTracker.java:startNewTask(1261))
- LaunchTaskAction: task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:45,994 WARN  conf.Configuration (Configuration.java:loadResource(893))
- C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a
attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
> 2008-04-02 19:54:46,087 WARN  fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052"
is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:46,228 INFO  mapred.TaskTracker (TaskTracker.java:reportProgress(1522))
- task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >

> 2008-04-02 19:54:47,618 INFO  mapred.JobClient (JobClient.java:runJob(995)) -  map 50%
reduce 16%
> 2008-04-02 19:54:49,180 INFO  dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064
Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:49,242 INFO  mapred.TaskTracker (TaskTracker.java:reportProgress(1522))
- task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >

> 2008-04-02 19:54:51,085 INFO  mapred.TaskInProgress (TaskInProgress.java:updateStatus(420))
- Error from task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not
found
> 	at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> 	at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:52,241 INFO  mapred.TaskTracker (TaskTracker.java:reportProgress(1522))
- task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >

> 2008-04-02 19:54:53,709 INFO  mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_1
done; removing files.
> 2008-04-02 19:54:53,787 INFO  mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084))
- Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:54:53,787 INFO  mapred.JobTracker (JobTracker.java:createTaskEntry(879))
- Adding task 'task_200804021954_0001_m_000000_2' to tip tip_200804021954_0001_m_000000, for
tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,787 INFO  mapred.JobTracker (JobTracker.java:removeMarkedTasks(974))
- Removed completed task 'task_200804021954_0001_m_000000_1' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:54:53,803 INFO  mapred.TaskTracker (TaskTracker.java:startNewTask(1261))
- LaunchTaskAction: task_200804021954_0001_m_000000_2
> 2008-04-02 19:54:53,834 WARN  conf.Configuration (Configuration.java:loadResource(893))
- C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a
attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
> 2008-04-02 19:54:53,897 INFO  mapred.TaskRunner (Task.java:discardTaskOutput(502)) -
Discarded output of task 'task_200804021954_0001_m_000000_1' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_1
> 2008-04-02 19:54:53,975 WARN  fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052"
is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:54:54,662 INFO  mapred.JobClient (JobClient.java:runJob(1046)) - Task Id
: task_200804021954_0001_m_000000_1, Status : FAILED
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,383 WARN  conf.Configuration (Configuration.java:loadResource(893))
- C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_1/job.xml:a
attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,415 INFO  jvm.JvmMetrics (JvmMetrics.java:init(67))
- Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:48,649 INFO  mapred.MapTask (MapTask.java:run(182))
- numReduceTasks: 1
> task_200804021954_0001_m_000000_1: 2008-04-02 19:54:49,180 WARN  mapred.TaskTracker (TaskTracker.java:main(2118))
- Error running child
> task_200804021954_0001_m_000000_1: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_1: 	at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_1: 	at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_1: 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_1: 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_1: 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:54:55,240 INFO  mapred.TaskTracker (TaskTracker.java:reportProgress(1522))
- task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >

> 2008-04-02 19:54:56,520 INFO  dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064
Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:54:57,223 INFO  mapred.TaskInProgress (TaskInProgress.java:updateStatus(420))
- Error from task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not
found
> 	at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> 	at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:01,065 INFO  mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_2
done; removing files.
> 2008-04-02 19:55:01,159 INFO  mapred.JobInProgress (JobInProgress.java:findNewMapTask(1084))
- Choosing data-local task tip_200804021954_0001_m_000000
> 2008-04-02 19:55:01,159 INFO  mapred.JobTracker (JobTracker.java:createTaskEntry(879))
- Adding task 'task_200804021954_0001_m_000000_3' to tip tip_200804021954_0001_m_000000, for
tracker 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,159 INFO  mapred.JobTracker (JobTracker.java:removeMarkedTasks(974))
- Removed completed task 'task_200804021954_0001_m_000000_2' from 'tracker_seemeight-dx.ds.corp.yahoo.com:localhost/127.0.0.1:2094'
> 2008-04-02 19:55:01,175 INFO  mapred.TaskTracker (TaskTracker.java:startNewTask(1261))
- LaunchTaskAction: task_200804021954_0001_m_000000_3
> 2008-04-02 19:55:01,206 WARN  conf.Configuration (Configuration.java:loadResource(893))
- C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/job.xml:a
attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
> 2008-04-02 19:55:01,253 INFO  mapred.TaskTracker (TaskTracker.java:reportProgress(1522))
- task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >

> 2008-04-02 19:55:01,284 INFO  mapred.TaskRunner (Task.java:discardTaskOutput(502)) -
Discarded output of task 'task_200804021954_0001_m_000000_2' - hdfs://localhost:2052/test/output/_temporary/_task_200804021954_0001_m_000000_2
> 2008-04-02 19:55:01,346 WARN  fs.FileSystem (FileSystem.java:fixName(153)) - "localhost:2052"
is a deprecated filesystem name. Use "hdfs://localhost:2052/" instead.
> 2008-04-02 19:55:01,690 INFO  mapred.JobClient (JobClient.java:runJob(1046)) - Task Id
: task_200804021954_0001_m_000000_2, Status : FAILED
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,708 WARN  conf.Configuration (Configuration.java:loadResource(893))
- C:/hudson/workspace/Hadoop-WindowsTest/trunk/build/test/mapred/local/1_0/taskTracker/jobcache/job_200804021954_0001/task_200804021954_0001_m_000000_2/job.xml:a
attempt to override final parameter: hadoop.tmp.dir;  Ignoring.
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,724 INFO  jvm.JvmMetrics (JvmMetrics.java:init(67))
- Initializing JVM Metrics with processName=MAP, sessionId=
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:55,974 INFO  mapred.MapTask (MapTask.java:run(182))
- numReduceTasks: 1
> task_200804021954_0001_m_000000_2: 2008-04-02 19:54:56,520 WARN  mapred.TaskTracker (TaskTracker.java:main(2118))
- Error running child
> task_200804021954_0001_m_000000_2: java.io.IOException: file file_tmpfile not found
> task_200804021954_0001_m_000000_2: 	at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> task_200804021954_0001_m_000000_2: 	at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> task_200804021954_0001_m_000000_2: 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> task_200804021954_0001_m_000000_2: 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> task_200804021954_0001_m_000000_2: 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:03,673 INFO  dfs.DataNode (DataNode.java:readBlock(1045)) - 127.0.0.1:2064
Served block blk_-5114990675256207114 to /127.0.0.1
> 2008-04-02 19:55:04,001 INFO  mapred.TaskInProgress (TaskInProgress.java:updateStatus(420))
- Error from task_200804021954_0001_m_000000_3: java.io.IOException: file file_tmpfile not
found
> 	at testshell.ExternalMapReduce.map(ExternalMapReduce.java:69)
> 	at testshell.ExternalMapReduce.map(ExternalMapReduce.java:42)
> 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:219)
> 	at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2113)
> 2008-04-02 19:55:07,250 INFO  mapred.TaskTracker (TaskTracker.java:reportProgress(1522))
- task_200804021954_0001_r_000000_0 0.16666667% reduce > copy (1 of 2 at 0.00 MB/s) >

> 2008-04-02 19:55:08,218 INFO  mapred.TaskRunner (MapTaskRunner.java:close(41)) - task_200804021954_0001_m_000000_3
done; removing files.
> 2008-04-02 19:55:08,281 INFO  mapred.TaskInProgress (TaskInProgress.java:incompleteSubTask(519))
- TaskInProgress tip_200804021954_0001_m_000000 has failed 4 times.
> 2008-04-02 19:55:08,281 INFO  mapred.JobInProgress (JobInProgress.java:addTrackerTaskFailure(741))
- TaskTracker at 'tracker_seemeight-dx.ds.corp.yahoo.com' turned 'flaky'
> 2008-04-02 19:55:08,281 INFO  mapred.JobInProgress (JobInProgress.java:failedTask(1556))
- Aborting job job_200804021954_0001
> 2008-04-02 19:55:08,312 INFO  dfs.StateChange (FSNamesystem.java:allocateBlock(1332))
- BLOCK* NameSystem.allocateBlock: /test/output/_logs/history/localhost_1207191264285_job_200804021954_0001_hadoopqa_external
job. blk_1520406918918394204
> 2008-04-02 19:55:08,328 INFO  dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving
block blk_1520406918918394204 src: /127.0.0.1:2241 dest: /127.0.0.1:2064
> 2008-04-02 19:55:08,343 INFO  dfs.DataNode (DataNode.java:writeBlock(1078)) - Receiving
block blk_1520406918918394204 src: /127.0.0.1:2242 dest: /127.0.0.1:2066
> 2008-04-02 19:55:08,359 INFO  dfs.StateChange (FSNamesystem.java:addStoredBlock(2833))
- BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2066 is added to blk_1520406918918394204
size 3263
> 2008-04-02 19:55:08,359 INFO  dfs.DataNode (DataNode.java:lastDataNodeRun(1943)) - Received
block blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,359 INFO  dfs.DataNode (DataNode.java:lastDataNodeRun(1961)) - PacketResponder
0 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,375 INFO  dfs.StateChange (FSNamesystem.java:addStoredBlock(2833))
- BLOCK* NameSystem.addStoredBlock: blockMap updated: 127.0.0.1:2064 is added to blk_1520406918918394204
size 3263
> 2008-04-02 19:55:08,375 INFO  dfs.DataNode (DataNode.java:run(2037)) - Received block
blk_1520406918918394204 of size 3263 from /127.0.0.1
> 2008-04-02 19:55:08,390 INFO  dfs.DataNode (DataNode.java:run(2095)) - PacketResponder
1 for block blk_1520406918918394204 terminating
> 2008-04-02 19:55:08,437 INFO  mapred.JobInProgress (JobInProgress.java:kill(1409)) -
Killing job 'job_200804021954_0001'

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message