hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Helmling <ghelml...@gmail.com>
Subject Re: [unit tests] small tests vs. others
Date Thu, 20 Oct 2011 21:50:22 GMT
+1 on small/medium/large

I have a test mvn repo up on people.apache.org that we can use for a
patched surefire build, if that helps.


On Thu, Oct 20, 2011 at 2:33 PM, Jonathan Gray <jgray@fb.com> wrote:
> +1 on small/medium/large.
>
>> -----Original Message-----
>> From: Todd Lipcon [mailto:todd@cloudera.com]
>> Sent: Thursday, October 20, 2011 1:49 PM
>> To: dev@hbase.apache.org
>> Subject: Re: [unit tests] small tests vs. others
>>
>> +1 on separating into small/medium/large. One of the requirements of
>> "small" should be that it can run parallel with other tests and in nofork mode.
>>
>> See SUREFIRE-329 for the necessary surefire patch. Until that's integrated
>> upstream we'd have to have our own patched surefire :(
>>
>> -Todd
>>
>> On Thu, Oct 20, 2011 at 1:46 PM, N Keywal <nkeywal@gmail.com> wrote:
>> > Hi,
>> >
>> > I've done a stat on the tests we have. Many of them are actually quite
>> > simple: There are 226 test classes in HBase.  100  of them does not
>> > use a cluster, and takes less than 2 seconds to execute. 24 do not use
>> > a cluster but need between 2 and 40 seconds.
>> >
>> > I tried to execute these with different maven settings.
>> > fork: once or always, i.e. all the tests are executed in the same JVM
>> > or there is a new JVM created for every test class parallel;
>> > deactivated or activated, with 3 threads and a per class
>> > parallelisation.
>> >
>> > The 100 tests executes for the whole maven stuff, including the
>> > initial dependency & task analysis fork always: 1m50s fork once: 30
>> > seconds
>> > parallel: 30s (=> no difference).
>> >
>> > It works quite well, except with TestLruBlockCache that I had to remove.
>> >
>> > I tried as well with the 24 'bigger' ones but no cluster, for the
>> > following
>> > results:
>> > fork always: 7m36s
>> > parallel: 2m32s: 3 times faster.
>> >
>> > However, I had to remove many of them, there are some conflicts that I
>> > have not solved (nor tried to).
>> >
>> > The fact that nearly 50% of the tests can be executed in 1 minutes is
>> > interesting imho.
>> >
>> > That makes me wonder if we should try to get a separation of the tests
>> > in 3 parts, small/medium/large as of
>> > http://googletesting.blogspot.com/2009/07/software-testing-categorizat
>> > ion.html
>> >
>> > small would be:
>> > - less than two seconds, can be executed in the same jvm
>> > - all of them: less than 2 minute
>> > - no cluster, nothing that can break the environment
>> > - run by the developers as a part of the build
>> >
>> > medium:
>> > - less than 40 seconds
>> > - all of them: would last less than 30 minutes, ideally 15 minutes
>> > - run in an isolated jvm
>> > - run explicitly by the developers before submitting a fix
>> >
>> > large:
>> > - run in an isolated jvm
>> > - Should last less than 45minutes
>> > - run on the central built or explicitly by the developers
>> > - break the built when it fails.
>> >
>> > Only one issue: it's not directly possible with maven, but there are
>> > known workarounds :-)
>> >
>> >
>> > What do you think?
>> >
>> >
>> > Below the tests set for the 120 tests that seems to work altogether.
>> >
>> >
>> > mvn test
>> > -
>> Dtest=org.apache.hadoop.hbase.avro.TestAvroUtil,org.apache.hadoop.hba
>> > se.client.TestAttributes,org.apache.hadoop.hbase.client.TestGet,org.ap
>> > ache.hadoop.hbase.client.TestOperation,org.apache.hadoop.hbase.client.
>> > TestResult,org.apache.hadoop.hbase.client.TestScan,org.apache.hadoop.h
>> >
>> base.coprocessor.TestRegionObserverStacking,org.apache.hadoop.hbase.fi
>> > lter.TestBitComparator,org.apache.hadoop.hbase.filter.TestColumnPagina
>> > tionFilter,org.apache.hadoop.hbase.filter.TestDependentColumnFilter,or
>> > g.apache.hadoop.hbase.filter.TestFilterList,org.apache.hadoop.hbase.fi
>> > lter.TestInclusiveStopFilter,org.apache.hadoop.hbase.filter.TestPageFi
>> > lter,org.apache.hadoop.hbase.filter.TestParseFilter,org.apache.hadoop.
>> > hbase.filter.TestPrefixFilter,org.apache.hadoop.hbase.filter.TestRando
>> >
>> mRowFilter,org.apache.hadoop.hbase.filter.TestSingleColumnValueExclude
>> > Filter,org.apache.hadoop.hbase.filter.TestSingleColumnValueFilter,org.
>> > apache.hadoop.hbase.io.hfile.slab.TestSlabCache,org.apache.hadoop.hbas
>> > e.io.hfile.slab.TestSlab,org.apache.hadoop.hbase.io.hfile.TestBlockCac
>> >
>> heColumnFamilySummary,org.apache.hadoop.hbase.io.hfile.TestCachedBloc
>> k
>> > Queue,org.apache.hadoop.hbase.io.hfile.TestFixedFileTrailer,org.apache
>> > .hadoop.hbase.io.hfile.TestHFileReaderV1,org.apache.hadoop.hbase.io.hf
>> > ile.TestHFile,org.apache.hadoop.hbase.io.hfile.TestHFileWriterV2,org.a
>> >
>> pache.hadoop.hbase.io.hfile.TestReseekTo,org.apache.hadoop.hbase.io.hf
>> > ile.TestSeekTo,org.apache.hadoop.hbase.io.TestHalfStoreFileReader,org.
>> >
>> apache.hadoop.hbase.io.TestHbaseObjectWritable,org.apache.hadoop.hbas
>> e
>> >
>> .io.TestHeapSize,org.apache.hadoop.hbase.io.TestImmutableBytesWritable
>> >
>> ,org.apache.hadoop.hbase.mapreduce.TestSimpleTotalOrderPartitioner,org
>> >
>> .apache.hadoop.hbase.mapreduce.TestTableSplit,org.apache.hadoop.hbase
>> .
>> >
>> master.TestClockSkewDetection,org.apache.hadoop.hbase.master.TestDea
>> dS
>> >
>> erver,org.apache.hadoop.hbase.master.TestDefaultLoadBalancer,org.apach
>> >
>> e.hadoop.hbase.master.TestMasterStatusServlet,org.apache.hadoop.hbase.
>> >
>> metrics.TestMetricsMBeanBase,org.apache.hadoop.hbase.monitoring.Test
>> Me
>> >
>> moryBoundedLogMessageBuffer,org.apache.hadoop.hbase.monitoring.Test
>> Tas
>> >
>> kMonitor,org.apache.hadoop.hbase.regionserver.handler.TestCloseRegionH
>> >
>> andler,org.apache.hadoop.hbase.regionserver.TestCompactSelection,org.a
>> > pache.hadoop.hbase.regionserver.TestExplicitColumnTracker,org.apache.h
>> >
>> adoop.hbase.regionserver.TestHRegionInfo,org.apache.hadoop.hbase.regio
>> >
>> nserver.TestKeyValueHeap,org.apache.hadoop.hbase.regionserver.TestKey
>> V
>> >
>> alueScanFixture,org.apache.hadoop.hbase.regionserver.TestKeyValueSkipL
>> >
>> istSet,org.apache.hadoop.hbase.regionserver.TestMemStoreLAB,org.apach
>> e
>> >
>> .hadoop.hbase.regionserver.TestMinVersions,org.apache.hadoop.hbase.reg
>> >
>> ionserver.TestQueryMatcher,org.apache.hadoop.hbase.regionserver.TestRe
>> > gionSplitPolicy,org.apache.hadoop.hbase.regionserver.TestResettingCoun
>> > ters,org.apache.hadoop.hbase.regionserver.TestRSStatusServlet,org.apac
>> >
>> he.hadoop.hbase.regionserver.TestScanDeleteTracker,org.apache.hadoop.h
>> >
>> base.regionserver.TestScanWildcardColumnTracker,org.apache.hadoop.hbas
>> >
>> e.regionserver.TestScanWithBloomError,org.apache.hadoop.hbase.regionse
>> >
>> rver.TestStoreScanner,org.apache.hadoop.hbase.regionserver.wal.TestHLo
>> >
>> gBench,org.apache.hadoop.hbase.regionserver.wal.TestHLogMethods,org.a
>> p
>> >
>> ache.hadoop.hbase.regionserver.wal.TestWALActionsListener,org.apache.h
>> >
>> adoop.hbase.rest.model.TestCellModel,org.apache.hadoop.hbase.rest.mod
>> e
>> >
>> l.TestCellSetModel,org.apache.hadoop.hbase.rest.model.TestColumnSchem
>> a
>> >
>> Model,org.apache.hadoop.hbase.rest.model.TestRowModelo,rg.apache.ha
>> doo
>> >
>> p.hbase.rest.model.TestScannerModel,org.apache.hadoop.hbase.rest.mod
>> el
>> >
>> .TestStorageClusterStatusModel,org.apache.hadoop.hbase.rest.model.Test
>> >
>> StorageClusterVersionModel,org.apache.hadoop.hbase.rest.model.TestTabl
>> >
>> eInfoModel,org.apache.hadoop.hbase.rest.model.TestTableListModel,org.a
>> >
>> pache.hadoop.hbase.rest.model.TestTableRegionModel,org.apache.hadoop
>> .h
>> >
>> base.rest.model.TestTableSchemaModel,org.apache.hadoop.hbase.rest.mo
>> de
>> > l.TestVersionModel,org.apache.hadoop.hbase.security.TestUser,org.apach
>> >
>> e.hadoop.hbase.TestAcidGuarantees,org.apache.hadoop.hbase.TestCompar
>> e,
>> >
>> org.apache.hadoop.hbase.TestHRegionLocation,org.apache.hadoop.hbase.T
>> e
>> >
>> stHServerAddress,org.apache.hadoop.hbase.TestHServerInfo,org.apache.ha
>> > doop.hbase.TestKeyValue,org.apache.hadoop.hbase.TestSerialization,org.
>> >
>> apache.hadoop.hbase.TestServerName,org.apache.hadoop.hbase.util.TestB
>> a
>> > se64,org.apache.hadoop.hbase.util.TestByteBloomFilter,org.apache.hadoo
>> > p.hbase.util.TestBytes,org.apache.hadoop.hbase.util.TestCompressionTes
>> > t,org.apache.hadoop.hbase.util.TestFSTableDescriptors,org.apache.hadoo
>> > p.hbase.util.TestHBaseFsckComparator,org.apache.hadoop.hbase.util.Test
>> >
>> IncrementingEnvironmentEdge,org.apache.hadoop.hbase.util.TestKeying,or
>> >
>> g.apache.hadoop.hbase.util.TestPoolMap,org.apache.hadoop.hbase.util.Te
>> >
>> stRegionSplitCalculator,org.apache.hadoop.hbase.zookeeper.TestHQuorum
>> P
>> >
>> eer,org.apache.hadoop.hbase.zookeeper.TestZooKeeperMainServerArg,org
>> .a
>> >
>> pache.hadoop.hbase.coprocessor.TestCoprocessorInterface,org.apache.had
>> > oop.hbase.executor.TestExecutorService,org.apache.hadoop.hbase.filter.
>> > TestColumnPrefixFilter,org.apache.hadoop.hbase.filter.TestFilter,org.a
>> > pache.hadoop.hbase.filter.TestMultipleColumnPrefixFilter,org.apache.ha
>> > doop.hbase.io.hfile.slab.TestSingleSizeCache,org.apache.hadoop.hbase.i
>> > o.hfile.TestCacheOnWrite,org.apache.hadoop.hbase.io.hfile.TestHFileBlo
>> > ckIndex,org.apache.hadoop.hbase.io.hfile.TestHFileBlock,org.apache.had
>> > oop.hbase.io.hfile.TestHFilePerformance,org.apache.hadoop.hbase.io.hfi
>> >
>> le.TestHFileSeek,org.apache.hadoop.hbase.ipc.TestDelayedRpc,org.apache
>> >
>> .hadoop.hbase.master.TestCatalogJanitor,org.apache.hadoop.hbase.region
>> >
>> server.TestAtomicOperation,org.apache.hadoop.hbase.regionserver.TestM
>> e
>> >
>> mStore,org.apache.hadoop.hbase.regionserver.TestReadWriteConsistencyC
>> o
>> > ntrol,org.apache.hadoop.hbase.regionserver.TestSeekOptimizations,org.a
>> >
>> pache.hadoop.hbase.regionserver.TestSplitTransaction,org.apache.hadoop
>> > .hbase.regionserver.TestStore,org.apache.hadoop.hbase.replication.Test
>> > ReplicationSource
>> >
>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>

Mime
View raw message