hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "nkeywal (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4737) Split the tests in small/medium/large; allow small tests to be ran in parallel within a single JVM
Date Sat, 05 Nov 2011 08:00:52 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13144626#comment-13144626
] 

nkeywal commented on HBASE-4737:
--------------------------------

bq. The failed tests from other groups weren't included in the above list.
What do you mean?
Note that surefire is buggy, and the results presented while running may be wrong. For example,
it can says that a class tests have 5 errors. When you look at the file it wrote, it says
that the 2 tests are ok, and in the class there are actually two tests methods, not five.
If you generate the report at the end with surefire-report it's fine however. I am going to
mention this in the script.
                
> Split the tests in small/medium/large; allow small tests to be ran in parallel within
a single JVM
> --------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-4737
>                 URL: https://issues.apache.org/jira/browse/HBASE-4737
>             Project: HBase
>          Issue Type: Improvement
>          Components: test
>    Affects Versions: 0.94.0
>            Reporter: nkeywal
>            Assignee: nkeywal
>            Priority: Minor
>         Attachments: 20111103_4737_pom.dummy.patch, 20111103_4737_pom.patch, 20111103_4737_pom.patch,
20111103_4737_pom.patch, 20111103_4737_pom.patch, 20111103_4737_pom.v2.patch, 20111103_4737_pom.v2.patch,
hbasetests.sh
>
>
> 1) Split the tests in 3 categories
>  - small: no cluster, less than 15s, can be run in parallel with other tests in a JVM
>  - medium: <45s, no flaky, useful to detect bugs immediatly
>  - large: remaining
>  
> 2) Allow to run a subset: developpers should need to run only small and medium before
submitting a patch
>  - will need a surefire patch, see http://jira.codehaus.org/browse/SUREFIRE-329
> Small is the default. All other tests will have to be marked Medium or Large with a JUnit
category.
> Proposed split:
> Small:122 classes, 479 methods, ~3 minutes when no //)
> Medium: 78 classes, 373 methods, ~23 minutes
> Large: 34 classes, 221 methods, ~60 minutes
> I will have to extract the methods that are today in large or medium but could be in
small (typically io.hfile.TestHFileBlock#testBlockHeapSize), it will be done in a second step
(and another JIRA).
> MEDIUM LIST (name; number of methods, time)
> org.apache.hadoop.hbase.avro.TestAvroServer	3	31.468
> org.apache.hadoop.hbase.catalog.TestCatalogTracker	8	4.174
> org.apache.hadoop.hbase.catalog.TestMetaReaderEditorNoCluster	1	3.888
> org.apache.hadoop.hbase.catalog.TestMetaReaderEditor	5	30.157
> org.apache.hadoop.hbase.client.replication.TestReplicationAdmin	1	0.762
> org.apache.hadoop.hbase.client.TestHCM	3	21.961
> org.apache.hadoop.hbase.client.TestHTablePool	18	26.274
> org.apache.hadoop.hbase.client.TestHTableUtil	2	16.997
> org.apache.hadoop.hbase.client.TestMetaMigrationRemovingHTD	3	24.629
> org.apache.hadoop.hbase.client.TestMetaScanner	1	16.365
> org.apache.hadoop.hbase.client.TestMultiParallel	10	34.077
> org.apache.hadoop.hbase.client.TestTimestampsFilter	3	27.547
> org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol	44	16.834
> org.apache.hadoop.hbase.coprocessor.TestClassLoading	5	31.346
> org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint	2	32.736
> org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort	1	13.874
> org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove	1	16.923
> org.apache.hadoop.hbase.coprocessor.TestMasterObserver	3	29.97
> org.apache.hadoop.hbase.coprocessor.TestRegionObserverBypass	2	14.976
> org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface	5	33.353
> org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort	1	16.596
> org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithRemove	1
18.183
> org.apache.hadoop.hbase.coprocessor.TestWALObserver	3	19.373
> org.apache.hadoop.hbase.filter.TestColumnRangeFilter	1	19.045
> org.apache.hadoop.hbase.io.hfile.slab.TestSingleSizeCache	5	24.294
> org.apache.hadoop.hbase.io.hfile.slab.TestSlabCache	7	19.818
> org.apache.hadoop.hbase.io.hfile.TestHFileBlock	7	25.226
> org.apache.hadoop.hbase.io.hfile.TestLruBlockCache	7	0.343
> org.apache.hadoop.hbase.mapreduce.TestImportTsv	8	40.391
> org.apache.hadoop.hbase.master.TestActiveMasterManager	2	0.724
> org.apache.hadoop.hbase.master.TestHMasterRPCException	1	1.17
> org.apache.hadoop.hbase.master.TestLogsCleaner	1	2.953
> org.apache.hadoop.hbase.master.TestMaster	1	18.918
> org.apache.hadoop.hbase.master.TestOpenedRegionHandler	2	20.57
> org.apache.hadoop.hbase.master.TestSplitLogManager	10	13.979
> org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion	3	21.675
> org.apache.hadoop.hbase.regionserver.handler.TestOpenRegionHandler	3	0.887
> org.apache.hadoop.hbase.regionserver.TestBlocksRead	4	1.42
> org.apache.hadoop.hbase.regionserver.TestCompoundBloomFilter	3	22.694
> org.apache.hadoop.hbase.regionserver.TestFSErrorsExposed	3	29.764
> org.apache.hadoop.hbase.regionserver.TestHRegion	57	28.552
> org.apache.hadoop.hbase.regionserver.TestMasterAddressManager	1	0.525
> org.apache.hadoop.hbase.regionserver.TestMultiColumnScanner	6	19.568
> org.apache.hadoop.hbase.regionserver.TestRpcMetrics	1	2.028
> org.apache.hadoop.hbase.regionserver.TestSeekOptimizations	6	3.031
> org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol	6	20.087
> org.apache.hadoop.hbase.regionserver.TestSplitLogWorker	5	2.062
> org.apache.hadoop.hbase.regionserver.TestStoreFileBlockCacheSummary	1	16.88
> org.apache.hadoop.hbase.regionserver.wal.TestLogRollAbort	1	22.545
> org.apache.hadoop.hbase.regionserver.wal.TestWALReplay	5	29.678
> org.apache.hadoop.hbase.replication.regionserver.TestReplicationSink	5	23.19
> org.apache.hadoop.hbase.replication.regionserver.TestReplicationSourceManager	1	0.918
> org.apache.hadoop.hbase.replication.TestReplicationPeer	1	25.518
> org.apache.hadoop.hbase.rest.client.TestRemoteAdmin	1	16.141
> org.apache.hadoop.hbase.rest.client.TestRemoteTable	3	17.827
> org.apache.hadoop.hbase.rest.TestGzipFilter	3	17.404
> org.apache.hadoop.hbase.rest.TestMultiRowResource	3	17.314
> org.apache.hadoop.hbase.rest.TestRowResource	11	17.73
> org.apache.hadoop.hbase.rest.TestScannerResource	5	26.868
> org.apache.hadoop.hbase.rest.TestScannersWithFilters	10	20.774
> org.apache.hadoop.hbase.rest.TestSchemaResource	2	22.441
> org.apache.hadoop.hbase.rest.TestStatusResource	2	14.884
> org.apache.hadoop.hbase.rest.TestTableResource	8	25.658
> org.apache.hadoop.hbase.rest.TestTransform	1	15.966
> org.apache.hadoop.hbase.rest.TestVersionResource	7	19.124
> org.apache.hadoop.hbase.TestGlobalMemStoreSize	1	18.154
> org.apache.hadoop.hbase.TestInfoServers	2	14.9
> org.apache.hadoop.hbase.TestMultiVersions	3	27.945
> org.apache.hadoop.hbase.thrift.TestThriftServer	1	20.072
> org.apache.hadoop.hbase.util.TestDefaultEnvironmentEdge	1	0.055
> org.apache.hadoop.hbase.util.TestEnvironmentEdgeManager	2	0.215
> org.apache.hadoop.hbase.util.TestFSUtils	2	19.988
> org.apache.hadoop.hbase.util.TestHBaseFsck	6	41.133
> org.apache.hadoop.hbase.util.TestIncrementingEnvironmentEdge	1	0.054
> org.apache.hadoop.hbase.util.TestMergeTable	1	27.125
> org.apache.hadoop.hbase.util.TestRegionSplitter	4	28.845
> org.apache.hadoop.hbase.zookeeper.TestZKTable	1	0.529
> org.apache.hadoop.hbase.zookeeper.TestZooKeeperNodeTracker	2	0.569
> LARGE
> org.apache.hadoop.hbase.catalog.TestCatalogTrackerOnCluster	1	84.463
> org.apache.hadoop.hbase.client.TestAdmin	33	369.833
> org.apache.hadoop.hbase.client.TestFromClientSide	49	154.41
> org.apache.hadoop.hbase.client.TestMultipleTimestamps	8	66.497
> org.apache.hadoop.hbase.client.TestScannerTimeout	4	47.309
> org.apache.hadoop.hbase.client.TestShell	1	69.65
> org.apache.hadoop.hbase.mapred.TestTableInputFormat	6	24.39
> org.apache.hadoop.hbase.mapred.TestTableMapReduce	1	63.846
> org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat	8	152.609
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery	3	74.628
> org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles	6	35.25
> org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan	11	516.619
> org.apache.hadoop.hbase.mapreduce.TestTableMapReduce	1	92.144
> org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed	1	42.761
> org.apache.hadoop.hbase.master.TestDistributedLogSplitting	4	112.606
> org.apache.hadoop.hbase.master.TestMasterFailover	4	76.658
> org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable	1	20.86
> org.apache.hadoop.hbase.master.TestMasterTransitions	3	22.043
> org.apache.hadoop.hbase.master.TestRestartCluster	2	36.325
> org.apache.hadoop.hbase.master.TestRollingRestart	1	54.738
> org.apache.hadoop.hbase.regionserver.TestEndToEndSplitTransaction	1	40.935
> org.apache.hadoop.hbase.regionserver.TestHRegionServerBulkLoad	1	47.073
> org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster	4	43.359
> org.apache.hadoop.hbase.regionserver.wal.TestHLogSplit	28	230.379
> org.apache.hadoop.hbase.regionserver.wal.TestHLog	9	59.889
> org.apache.hadoop.hbase.regionserver.wal.TestLogRolling	3	310.537
> org.apache.hadoop.hbase.replication.TestMasterReplication	2	75.346
> org.apache.hadoop.hbase.replication.TestMultiSlaveReplication	1	47.873
> org.apache.hadoop.hbase.replication.TestReplication	7	163.158
> org.apache.hadoop.hbase.TestFullLogReconstruction	1	57.742
> org.apache.hadoop.hbase.TestHBaseTestingUtility	7	73.654
> org.apache.hadoop.hbase.TestRegionRebalancing	1	58.25
> org.apache.hadoop.hbase.TestZooKeeper	7	86.267
> org.apache.hadoop.hbase.util.TestMergeTool	1	257.223
> SMALL
> Everyting else

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message