hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-8303) Clean up hadoop-streaming
Date Sun, 22 Apr 2012 20:57:33 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-8303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Harsh J updated HADOOP-8303:
----------------------------

    Attachment: 0008-MapReduce-Streaming-Cleanup.patch

A few things:

# Looks like even Hadoop-Common QA does not pick up hadoop-tools changes.
# The eclipse's auto unused import fixer removed RecRecord imports which caused TestIO not
to compile. I fixed those manually in this new patch.
# TestStreaming had a change in original patch that caused it to lose info about the running
jobs and such. I tweaked the change to use a non ToolRunner approach, and TestStreamingCounters/Combiner
now work nicely again - minus deprecation warnings.

Since Hadoop QA bot can't run the tests, here is the test result manually run, with patch
applied:

{{cd hadoop-tools/hadoop-streaming; mvn clean test}}

{code}
-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.hadoop.streaming.TestAutoInputFormat
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.663 sec
Running org.apache.hadoop.streaming.TestClassWithNoPackage
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.261 sec
Running org.apache.hadoop.streaming.TestDumpTypedBytes
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.21 sec
Running org.apache.hadoop.streaming.TestFileArgs
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 19.49 sec
Running org.apache.hadoop.streaming.TestGzipInput
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.188 sec
Running org.apache.hadoop.streaming.TestLoadTypedBytes
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.486 sec
Running org.apache.hadoop.streaming.TestMRFramework
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.265 sec
Running org.apache.hadoop.streaming.TestMultipleArchiveFiles
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 22.727 sec
Running org.apache.hadoop.streaming.TestMultipleCachefiles
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 25.324 sec
Running org.apache.hadoop.streaming.TestRawBytesStreaming
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.07 sec
Running org.apache.hadoop.streaming.TestStreamAggregate
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.062 sec
Running org.apache.hadoop.streaming.TestStreamDataProtocol
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.058 sec
Running org.apache.hadoop.streaming.TestStreaming
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.165 sec
Running org.apache.hadoop.streaming.TestStreamingBackground
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.09 sec
Running org.apache.hadoop.streaming.TestStreamingBadRecords
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.769 sec
Running org.apache.hadoop.streaming.TestStreamingCombiner
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.21 sec
Running org.apache.hadoop.streaming.TestStreamingCounters
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.191 sec
Running org.apache.hadoop.streaming.TestStreamingExitStatus
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.031 sec
Running org.apache.hadoop.streaming.TestStreamingFailure
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.788 sec
Running org.apache.hadoop.streaming.TestStreamingKeyValue
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.324 sec
Running org.apache.hadoop.streaming.TestStreamingOutputKeyValueTypes
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.626 sec
Running org.apache.hadoop.streaming.TestStreamingSeparator
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.068 sec
Running org.apache.hadoop.streaming.TestStreamingStderr
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.579 sec
Running org.apache.hadoop.streaming.TestStreamingTaskLog
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 15.659 sec
Running org.apache.hadoop.streaming.TestStreamJob
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.761 sec
Running org.apache.hadoop.streaming.TestStreamReduceNone
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.146 sec
Running org.apache.hadoop.streaming.TestStreamXmlMultipleRecords
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.238 sec
Running org.apache.hadoop.streaming.TestStreamXmlRecordReader
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.152 sec
Running org.apache.hadoop.streaming.TestSymLink
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.418 sec
Running org.apache.hadoop.streaming.TestTypedBytesStreaming
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.146 sec
Running org.apache.hadoop.streaming.TestUnconsumedInput
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.139 sec
Running org.apache.hadoop.typedbytes.TestIO
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.166 sec
Running org.apache.hadoop.typedbytes.TestTypedBytesWritable
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.032 sec

Results :

Tests run: 59, Failures: 0, Errors: 0, Skipped: 0
{code}
                
> Clean up hadoop-streaming
> -------------------------
>
>                 Key: HADOOP-8303
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8303
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: build
>    Affects Versions: 3.0.0
>            Reporter: Harsh J
>            Assignee: Harsh J
>            Priority: Minor
>         Attachments: 0008-MapReduce-Streaming-Cleanup.patch, 0008-MapReduce-Streaming-Cleanup.patch
>
>
> Clean up a bunch of existing javac warnings in hadoop-streaming module.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message