hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12331) Shorten the mob snapshot unit tests
Date Wed, 29 Oct 2014 23:21:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14189261#comment-14189261
] 

Jonathan Hsieh commented on HBASE-12331:
----------------------------------------

I did a little more digging -- there are a few general requests

1) some tests like TestCloneSnapshotFromclientWithRegionReplicas/TestMobCloneSnapshotFromClientWithRegionReplicas
seems like overkill.  I'd think that read replicas are orthogonal to snapshots or have specific
tests for the combination of features instead of running the exact same tests.  If you want
to run the same tests it shoudl be just a configuration of an IT rig.

2) Could we refactor combine these so that we don't spin up clusters as often?  Maybe parameterize
tests?   Each of the MobSnashot tests is essentially a modified clone of the original snapshot
test, and we've added a lot of code for that.

3) For the tests that take >100s, can we write less data or do less setup so that they
finish in less time?  Could we set lower thresholds so that the code paths are exercise but
we write less data?


> Shorten the mob snapshot unit tests
> -----------------------------------
>
>                 Key: HBASE-12331
>                 URL: https://issues.apache.org/jira/browse/HBASE-12331
>             Project: HBase
>          Issue Type: Sub-task
>          Components: mob
>    Affects Versions: hbase-11339
>            Reporter: Jonathan Hsieh
>             Fix For: hbase-11339
>
>
> The mob snapshot patch introduced a whole log of tests that take a long time to run and
would be better as integration tests.
> {code}
> -------------------------------------------------------
>  T E S T S
> -------------------------------------------------------
> Running org.apache.hadoop.hbase.client.TestMobRestoreSnapshotFromClientWithRegionReplicas
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 394.803 sec - in org.apache.hadoop.hbase.client.TestMobRestoreSnapshotFromClientWithRegionReplicas
> Running org.apache.hadoop.hbase.client.TestMobRestoreSnapshotFromClient
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 212.377 sec - in org.apache.hadoop.hbase.client.TestMobRestoreSnapshotFromClient
> Running org.apache.hadoop.hbase.client.TestMobSnapshotFromClientWithRegionReplicas
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 49.463 sec - in org.apache.hadoop.hbase.client.TestMobSnapshotFromClientWithRegionReplicas
> Running org.apache.hadoop.hbase.client.TestMobSnapshotFromClient
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 46.724 sec - in org.apache.hadoop.hbase.client.TestMobSnapshotFromClient
> Running org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 204.03 sec - in org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClient
> Running org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClientWithRegionReplicas
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 214.052 sec - in org.apache.hadoop.hbase.client.TestMobCloneSnapshotFromClientWithRegionReplicas
> Running org.apache.hadoop.hbase.client.TestMobSnapshotCloneIndependence
> Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 105.139 sec - in org.apache.hadoop.hbase.client.TestMobSnapshotCloneIndependence
> Running org.apache.hadoop.hbase.regionserver.TestMobStoreScanner
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.42 sec - in org.apache.hadoop.hbase.regionserver.TestMobStoreScanner
> Running org.apache.hadoop.hbase.regionserver.TestDeleteMobTable
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.136 sec - in org.apache.hadoop.hbase.regionserver.TestDeleteMobTable
> Running org.apache.hadoop.hbase.regionserver.TestHMobStore
> Tests run: 7, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.09 sec - in org.apache.hadoop.hbase.regionserver.TestHMobStore
> Running org.apache.hadoop.hbase.regionserver.TestMobCompaction
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.629 sec - in org.apache.hadoop.hbase.regionserver.TestMobCompaction
> Running org.apache.hadoop.hbase.mob.TestCachedMobFile
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.301 sec - in org.apache.hadoop.hbase.mob.TestCachedMobFile
> Running org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepJob
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 14.752 sec - in org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepJob
> Running org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepReducer
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.276 sec - in org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepReducer
> Running org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepMapper
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 13.46 sec - in org.apache.hadoop.hbase.mob.mapreduce.TestMobSweepMapper
> Running org.apache.hadoop.hbase.mob.mapreduce.TestMobSweeper
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 173.05 sec - in org.apache.hadoop.hbase.mob.mapreduce.TestMobSweeper
> Running org.apache.hadoop.hbase.mob.TestMobDataBlockEncoding
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 20.86 sec - in org.apache.hadoop.hbase.mob.TestMobDataBlockEncoding
> Running org.apache.hadoop.hbase.mob.TestExpiredMobFileCleaner
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.029 sec - in org.apache.hadoop.hbase.mob.TestExpiredMobFileCleaner
> Running org.apache.hadoop.hbase.mob.TestMobFile
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.562 sec - in org.apache.hadoop.hbase.mob.TestMobFile
> Running org.apache.hadoop.hbase.mob.TestMobFileCache
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.173 sec - in org.apache.hadoop.hbase.mob.TestMobFileCache
> Running org.apache.hadoop.hbase.mob.TestDefaultMobStoreFlusher
> Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 16.586 sec - in org.apache.hadoop.hbase.mob.TestDefaultMobStoreFlusher
> Running org.apache.hadoop.hbase.mob.TestMobFileName
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.034 sec - in org.apache.hadoop.hbase.mob.TestMobFileName
> Running org.apache.hadoop.hbase.snapshot.TestMobSecureExportSnapshot
> Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 707.528 sec - in org.apache.hadoop.hbase.snapshot.TestMobSecureExportSnapshot
> Running org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot
> Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 492.225 sec - in org.apache.hadoop.hbase.snapshot.TestMobExportSnapshot
> Running org.apache.hadoop.hbase.snapshot.TestMobRestoreSnapshotHelper
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.818 sec - in org.apache.hadoop.hbase.snapshot.TestMobRestoreSnapshotHelper
> Running org.apache.hadoop.hbase.snapshot.TestMobFlushSnapshotFromClient
> Tests run: 9, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 83.69 sec - in org.apache.hadoop.hbase.snapshot.TestMobFlushSnapshotFromClient
> Running org.apache.hadoop.hbase.snapshot.TestMobRestoreFlushSnapshotFromClient
> Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 144.088 sec - in org.apache.hadoop.hbase.snapshot.TestMobRestoreFlushSnapshotFromClient
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message