hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5847) Streaming unit tests failing for a while on trunk
Date Sun, 24 May 2009 11:06:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12712540#action_12712540
] 

Steve Loughran commented on HADOOP-5847:
----------------------------------------

I'm running these tests at home (OSX) , and am seeing a failure
   [junit] Tests run: 4, Failures: 1, Errors: 0, Time elapsed: 13.164 sec
    [junit] Test org.apache.hadoop.streaming.TestStreamingExitStatus FAILED

Looks like it is just an OOM event though.

09/05/24 12:00:11 INFO streaming.PipeMapRed: R/W/S=1/0/0 in:NA [rec/s] out:NA [rec/s]
09/05/24 12:00:11 INFO streaming.PipeMapRed: Records R/W=1/1
09/05/24 12:00:11 INFO streaming.PipeMapRed: MRErrorThread done
09/05/24 12:00:11 WARN mapred.LocalJobRunner: job_local_0002
java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1
	at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:366)
	at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:568)
	at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:135)
	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57)
	at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:360)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:309)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:178)
09/05/24 12:00:11 INFO streaming.StreamJob:  map 0%  reduce 0%
09/05/24 12:00:11 INFO streaming.StreamJob: Job running in-process (local Hadoop)
09/05/24 12:00:11 ERROR streaming.StreamJob: Job not Successful!
09/05/24 12:00:11 INFO streaming.StreamJob: killJob...
09/05/24 12:00:11 WARN streaming.StreamJob: -jobconf option is deprecated, please use -D instead.
09/05/24 12:00:11 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker,
sessionId= - already initialized
09/05/24 12:00:11 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
09/05/24 12:00:12 WARN mapred.JobClient: No job jar file set.  User classes may not be found.
See JobConf(Class) or JobConf#setJar(String).
09/05/24 12:00:12 INFO mapred.FileInputFormat: Total input paths to process : 1
09/05/24 12:00:13 INFO mapred.FileInputFormat: Total input paths to process : 1
09/05/24 12:00:13 INFO streaming.StreamJob: getLocalDirs(): [/Users/slo/Java/Apache/hadoop-branches/hadoop-5847/build/contrib/streaming/test/mapred/local]
09/05/24 12:00:13 INFO streaming.StreamJob: Running job: job_local_0003
09/05/24 12:00:13 INFO streaming.StreamJob: Job running in-process (local Hadoop)
09/05/24 12:00:13 INFO mapred.MapTask: numReduceTasks: 1
09/05/24 12:00:13 INFO mapred.MapTask: io.sort.mb = 100
09/05/24 12:00:13 WARN mapred.LocalJobRunner: job_local_0003
java.lang.OutOfMemoryError: Java heap space
	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:689)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:352)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:309)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:178)
09/05/24 12:00:14 INFO streaming.StreamJob:  map 0%  reduce 0%
09/05/24 12:00:14 INFO streaming.StreamJob: Job running in-process (local Hadoop)
09/05/24 12:00:14 ERROR streaming.StreamJob: Job not Successful!
09/05/24 12:00:14 INFO streaming.StreamJob: killJob...
09/05/24 12:00:14 WARN streaming.StreamJob: -jobconf option is deprecated, please use -D instead.
09/05/24 12:00:14 INFO jvm.JvmMetrics: Cannot initialize JVM Metrics with processName=JobTracker,
sessionId= - already initialized
09/05/24 12:00:14 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments.
Applications should implement Tool for the same.
09/05/24 12:00:14 WARN mapred.JobClient: No job jar file set.  User classes may not be found.
See JobConf(Class) or JobConf#setJar(String).
09/05/24 12:00:14 INFO mapred.FileInputFormat: Total input paths to process : 1
09/05/24 12:00:15 INFO streaming.StreamJob: getLocalDirs(): [/Users/slo/Java/Apache/hadoop-branches/hadoop-5847/build/contrib/streaming/test/mapred/local]
09/05/24 12:00:15 INFO streaming.StreamJob: Running job: job_local_0004
09/05/24 12:00:15 INFO streaming.StreamJob: Job running in-process (local Hadoop)
09/05/24 12:00:15 INFO mapred.FileInputFormat: Total input paths to process : 1
09/05/24 12:00:15 INFO mapred.MapTask: numReduceTasks: 1
09/05/24 12:00:15 INFO mapred.MapTask: io.sort.mb = 100
09/05/24 12:00:16 WARN mapred.LocalJobRunner: job_local_0004
java.lang.OutOfMemoryError: Java heap space
	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:689)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:352)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:309)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:178)
09/05/24 12:00:16 INFO streaming.StreamJob:  map 0%  reduce 0%
09/05/24 12:00:16 INFO streaming.StreamJob: Job running in-process (local Hadoop)
09/05/24 12:00:16 ERROR streaming.StreamJob: Job not Successful!
09/05/24 12:00:16 INFO streaming.StreamJob: killJob...
------------- ---------------- ---------------



> Streaming unit tests failing for a while on trunk
> -------------------------------------------------
>
>                 Key: HADOOP-5847
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5847
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: contrib/streaming
>            Reporter: Hemanth Yamijala
>            Assignee: Giridharan Kesavan
>         Attachments: HADOOP-5847.patch
>
>
> http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/337/testReport/
shows some failed tests:
> org.apache.hadoop.streaming.TestMultipleCachefiles.testMultipleCachefiles
> org.apache.hadoop.streaming.TestStreamingBadRecords.testSkip
> org.apache.hadoop.streaming.TestStreamingBadRecords.testNarrowDown
> org.apache.hadoop.streaming.TestSymLink.testSymLink

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message