hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Olga Natkovich" <ol...@yahoo-inc.com>
Subject RE: Slow unit testing
Date Mon, 12 May 2008 16:45:38 GMT
We do see similar slowdown testing with hadoop 0.17 and still trying to
figure out why this is happening. I wonder if this could be related. 

> -----Original Message-----
> From: Alan Gates [mailto:gates@yahoo-inc.com] 
> Sent: Monday, May 12, 2008 9:41 AM
> To: pig-dev@incubator.apache.org
> Subject: Re: Slow unit testing
> 
> Sorry, I should have been more clear.  Time results similar 
> to Olga's, not Pi's.
> 
> Alan.
> 
> Alan Gates wrote:
> > I run the unit tests on a mac laptop and am seeing similar 
> time results.
> >
> > Alan.
> >
> > Olga Natkovich wrote:
> >> Pi,
> >>
> >> Inline my stdout content. Attached test output files. I ran this 
> >> against latest trunk on a linux box.
> >>
> >> test-local:
> >>
> >> testOneMode:
> >>     [mkdir] Created dir:
> >> /home/olgan/src/pig-apache3/trunk/build/test/logs-local
> >>     [junit] Running org.apache.pig.test.TestAlgebraicEval
> >>     [junit] Tests run: 5, Failures: 0, Errors: 0, Time 
> elapsed: 2.93 sec
> >>     [junit] Running org.apache.pig.test.TestBinaryStorage
> >>     [junit] Tests run: 2, Failures: 0, Errors: 0, Time 
> elapsed: 0.662 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestBuiltin
> >>     [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: 
> >> 3.849 sec
> >>     [junit] Running org.apache.pig.test.TestCmdLineParser
> >>     [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 
> >> 0.011 sec
> >>     [junit] Running org.apache.pig.test.TestCombiner
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time 
> elapsed: 0.421 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestCompressedFiles
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time 
> elapsed: 0.337 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestCustomSlicer
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time 
> elapsed: 0.214 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestDataBag
> >>     [junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 
> >> 1.461 sec
> >>     [junit] Running org.apache.pig.test.TestDataModel
> >>     [junit] Tests run: 4, Failures: 0, Errors: 0, Time 
> elapsed: 0.126 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestEvalPipeline
> >>     [junit] Tests run: 7, Failures: 0, Errors: 0, Time 
> elapsed: 5.035 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestExGenCogroup
> >>     [junit] Tests run: 4, Failures: 0, Errors: 0, Time 
> elapsed: 1.246 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestExGenEval
> >>     [junit] Tests run: 3, Failures: 0, Errors: 0, Time 
> elapsed: 1.172 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestFilterOpNumeric
> >>     [junit] Tests run: 8, Failures: 0, Errors: 0, Time 
> elapsed: 1.76 sec
> >>     [junit] Running org.apache.pig.test.TestFilterOpString
> >>     [junit] Tests run: 6, Failures: 0, Errors: 0, Time 
> elapsed: 1.385 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestGrunt
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time 
> elapsed: 0.238 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestInfixArithmetic
> >>     [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 
> >> 57.608 sec
> >>     [junit] Running org.apache.pig.test.TestLogicalPlanBuilder
> >>     [junit] Tests run: 49, Failures: 0, Errors: 0, Time 
> elapsed: 1.04 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestMapReduce
> >>     [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 
> >> 73.303 sec
> >>     [junit] Running 
> org.apache.pig.test.TestMapReduceResultRecycling
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time 
> elapsed: 1.542 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestParamSubPreproc
> >>     [junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed: 
> >> 0.633 sec
> >>     [junit] Running org.apache.pig.test.TestParser
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time 
> elapsed: 0.206 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestPigContext
> >>     [junit] Tests run: 3, Failures: 0, Errors: 0, Time 
> elapsed: 0.806 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestPigFile
> >>     [junit] Tests run: 3, Failures: 0, Errors: 0, Time 
> elapsed: 3.344 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestPigScriptParser
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time 
> elapsed: 0.441 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestPigServer
> >>     [junit] Tests run: 4, Failures: 0, Errors: 0, Time 
> elapsed: 5.76 sec
> >>     [junit] Running org.apache.pig.test.TestPigSplit
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time 
> elapsed: 0.546 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestReversibleLoadStore
> >>     [junit] Tests run: 2, Failures: 0, Errors: 0, Time 
> elapsed: 0.622 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestStore
> >>     [junit] Tests run: 3, Failures: 0, Errors: 0, Time 
> elapsed: 2.139 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestStreaming
> >>     [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 
> >> 1.074 sec
> >>
> >> test-mapreduce:
> >>
> >> testOneMode:
> >>     [mkdir] Created dir:
> >> /home/olgan/src/pig-apache3/trunk/build/test/logs-mapreduce
> >>     [junit] Running org.apache.pig.test.TestAlgebraicEval
> >>     [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 
> >> 74.738 sec
> >>     [junit] Running org.apache.pig.test.TestBinaryStorage
> >>     [junit] Tests run: 2, Failures: 0, Errors: 0, Time 
> elapsed: 36.99 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestBuiltin
> >>     [junit] Tests run: 21, Failures: 0, Errors: 0, Time elapsed: 
> >> 4.513 sec
> >>     [junit] Running org.apache.pig.test.TestCmdLineParser
> >>     [junit] Tests run: 17, Failures: 0, Errors: 0, Time elapsed: 
> >> 0.034 sec
> >>     [junit] Running org.apache.pig.test.TestCombiner
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time 
> elapsed: 24.9 sec
> >>     [junit] Running org.apache.pig.test.TestCompressedFiles
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 
> >> 29.658 sec
> >>     [junit] Running org.apache.pig.test.TestCustomSlicer
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 
> >> 90.949 sec
> >>     [junit] Running org.apache.pig.test.TestDataBag
> >>     [junit] Tests run: 18, Failures: 0, Errors: 0, Time elapsed: 
> >> 1.605 sec
> >>     [junit] Running org.apache.pig.test.TestDataModel
> >>     [junit] Tests run: 4, Failures: 0, Errors: 0, Time 
> elapsed: 0.194 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestEvalPipeline
> >>     [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed: 
> >> 203.826 sec
> >>     [junit] Running org.apache.pig.test.TestExGenCogroup
> >>     [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 
> >> 15.116 sec
> >>     [junit] Running org.apache.pig.test.TestExGenEval
> >>     [junit] Tests run: 3, Failures: 0, Errors: 0, Time 
> elapsed: 8.609 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestFilterOpNumeric
> >>     [junit] Tests run: 8, Failures: 0, Errors: 0, Time elapsed: 
> >> 104.898 sec
> >>     [junit] Running org.apache.pig.test.TestFilterOpString
> >>     [junit] Tests run: 6, Failures: 0, Errors: 0, Time elapsed: 
> >> 83.079 sec
> >>     [junit] Running org.apache.pig.test.TestGrunt
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time 
> elapsed: 0.296 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestInfixArithmetic
> >>     [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 
> >> 50.802 sec
> >>     [junit] Running org.apache.pig.test.TestLogicalPlanBuilder
> >>     [junit] Tests run: 49, Failures: 0, Errors: 0, Time elapsed: 
> >> 1.328 sec
> >>     [junit] Running org.apache.pig.test.TestMapReduce
> >>     [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed: 
> >> 52.169 sec
> >>     [junit] Running 
> org.apache.pig.test.TestMapReduceResultRecycling
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 
> >> 43.827 sec
> >>     [junit] Running org.apache.pig.test.TestParamSubPreproc
> >>     [junit] Tests run: 28, Failures: 0, Errors: 0, Time elapsed: 
> >> 0.724 sec
> >>     [junit] Running org.apache.pig.test.TestParser
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time 
> elapsed: 8.08 sec
> >>     [junit] Running org.apache.pig.test.TestPigContext
> >>     [junit] Tests run: 3, Failures: 0, Errors: 0, Time 
> elapsed: 1.209 
> >> sec
> >>     [junit] Running org.apache.pig.test.TestPigFile
> >>     [junit] Tests run: 3, Failures: 0, Errors: 0, Time 
> elapsed: 0.95 sec
> >>     [junit] Running org.apache.pig.test.TestPigScriptParser
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time 
> elapsed: 0.26 sec
> >>     [junit] Running org.apache.pig.test.TestPigServer
> >>     [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed: 
> >> 10.313 sec
> >>     [junit] Running org.apache.pig.test.TestPigSplit
> >>     [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 
> >> 109.971 sec
> >>     [junit] Running org.apache.pig.test.TestReversibleLoadStore
> >>     [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed: 
> >> 11.803 sec
> >>     [junit] Running org.apache.pig.test.TestStore
> >>     [junit] Tests run: 3, Failures: 0, Errors: 0, Time elapsed: 
> >> 125.27 sec
> >>     [junit] Running org.apache.pig.test.TestStreaming
> >>     [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed: 
> >> 142.835 sec
> >>
> >> test:
> >>
> >> BUILD SUCCESSFUL
> >> Total time: 26 minutes 4 seconds
> >> ucdev3:~/src/pig-apache3/trunk> cd
> >> ~/src/pig-yahoo/trunk/pig/conf/piglet/hadoop-site.xml
> 

Mime
View raw message