ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Fernandez" <afernan...@hortonworks.com>
Subject Re: Review Request 41569: AMBARI-14439. Categorize Ambari unit tests with Slow/Fast to run only desired group
Date Mon, 28 Dec 2015 23:08:24 GMT


> On Dec. 19, 2015, 10:18 p.m., Jonathan Hurley wrote:
> > I think we need to drop the notion of fast vs non-fast. I know that Nate has been
working on re-using the same persistence unit between tests (each test gets its own cluster,
own unique hosts, etc) ... and this has dramatically increased performance. 
> > 
> > With that said, I think we can still categorize and label our tests. But I think
we should take a difference approach.
> > 
> > It's cool to group them logically like Alerts, Kerberos, ORM, Upgrade, etc. 
> > 
> > But we should also create groupings based on what the tests are doing. Something
like:
> > SmallTest - accesses only mocks, most used for direct unit tests of single functional
aspects
> > MediumTest - works with mocks and DB, sets up cluster and some data for a single
unit of test
> > LargeTest - an end-to-end style test (like testing an upgrade or downgrade)
> > 
> > Even more, we can group them for quick runs from the command by mixing and matching
above logical groupings. Consider:
> > MAT (Minimum Acceptance Tests): Basic functionality that must pass (like cluster
creation, adding services, pure ORM)
> > SRG (Short Regression): A mixture of tests from above logical units (Alerts, Kerberos,
Upgrade) which don't take too long
> > LRG (Long Regression): A mixture of tests from above logical units (Alerts, Kerberos,
Upgrade) which are known to take a long time
> 
> Nate Cole wrote:
>     +1 to this.
> 
> Dmitro Lisnichenko wrote:
>     Are not SmallTest/MediumTest/LargeTest something like unit/integration/functional
tests? I'd say that a big part of our tests consists of over-detailed functional tests, that
tend to test too much and that's why break on any implementation change like mock method invocation
count. So I'd strongly +1 to clear separation of roles you are proposing. Behaviour
>     
>     Also, it would be great to have some commonly-used mock implementations like cluster
with few hosts and services as a separate class, so we could skip defining something like
createCluster() at every test suite.
> 
> Dmitro Lisnichenko wrote:
>     * I mean that functional/integration tests should test only high-level behaviour,
and mock out any logic that is covered by unit tests. So tests are more loosely coupled
> 
> Jayush Luniya wrote:
>     +1 on the suggestions above.

I like these suggestions. Since performance can always change, I'm ok with renaming them as
Unit Test - Very simple mocks, in the order of 100 ms or faster
Integration Test - multiple classes being tested together, in the order of 1000 ms or faster
Functional Test - heavy lifting that tests features (anything that has to setup a DB, add
cluster, services, hosts, perform RUs, etc). Some of these can take less than 1000 ms, but
for sure if it takes longer than 1000 ms it is likely in this category.

In additional, I will tag them with the feature, e.g., Stack Upgrade, Security (Kerberos),
Blueprints, etc.
This way, we can use the test runner to be very specific, e.g., Stack Upgrade && Unit
Test.


- Alejandro


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/41569/#review111389
-----------------------------------------------------------


On Dec. 18, 2015, 10:26 p.m., Alejandro Fernandez wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/41569/
> -----------------------------------------------------------
> 
> (Updated Dec. 18, 2015, 10:26 p.m.)
> 
> 
> Review request for Ambari, Aravindan Vijayan, Dmytro Grinenko, Di Li, Dmitro Lisnichenko,
Jaimin Jetly, Jonathan Hurley, Jayush Luniya, Nate Cole, Robert Levas, Robert Nettleton, Nahappan
Somasundaram, Sumit Mohanty, Srimanth Gunturi, Swapan Shridhar, Sid Wagle, and Yusaku Sako.
> 
> 
> Bugs: AMBARI-14439
>     https://issues.apache.org/jira/browse/AMBARI-14439
> 
> 
> Repository: ambari
> 
> 
> Description
> -------
> 
> Categorize the unit tests with wither SlowTest, FastTest, and some other group based
on the area.
> This will allow us to run a test suite that matches a given group.
> 
>     mvn clean test -P ${profile_id}
>     mvn clean -Drat.ignoreErrors=true test -P FastTest > fast-tests.txt
>     mvn clean -Drat.ignoreErrors=true test -P NonFastTest > nonfast-tests.txt
> 
> 
> Diffs
> -----
> 
>   ambari-server/pom.xml 56d9e44 
>   ambari-server/src/test/java/org/apache/ambari/server/AlertTest.java PRE-CREATION 
>   ambari-server/src/test/java/org/apache/ambari/server/AmbariUpgradeTest.java PRE-CREATION

>   ambari-server/src/test/java/org/apache/ambari/server/BlueprintTest.java PRE-CREATION

>   ambari-server/src/test/java/org/apache/ambari/server/FastTest.java PRE-CREATION 
>   ambari-server/src/test/java/org/apache/ambari/server/KerberosTest.java PRE-CREATION

>   ambari-server/src/test/java/org/apache/ambari/server/SlowTest.java PRE-CREATION 
>   ambari-server/src/test/java/org/apache/ambari/server/StackUpgradeTest.java PRE-CREATION

>   ambari-server/src/test/java/org/apache/ambari/server/checks/ClientRetryPropertyCheckTest.java
7b8239c 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/ComponentsInstallationCheckTest.java
450d74e 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/ConfigurationMergeCheckTest.java
68a0522 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/HiveDynamicServiceDiscoveryCheckTest.java
cdf13eb 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/HiveMultipleMetastoreCheckTest.java
16f383a 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/HostMaintenanceModeCheckTest.java
0e14376 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/HostsHeartbeatCheckTest.java
cc2c276 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/HostsMasterMaintenanceCheckTest.java
9fcb319 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/HostsRepositoryVersionCheckTest.java
4529554 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/InstallPackagesCheckTest.java
080ca3a 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/MapReduce2JobHistoryStatePreservingCheckTest.java
bfe0c3e 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/RangerPasswordCheckTest.java
afa3789 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/SecondaryNamenodeDeletedCheckTest.java
e2617bf 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/ServicesMaintenanceModeCheckTest.java
a941b7a 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/ServicesMapReduceDistributedCacheCheckTest.java
22f2b1b 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/ServicesNamenodeHighAvailabilityCheckTest.java
abe7abe 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/ServicesNamenodeTruncateCheckTest.java
87d4167 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/ServicesUpCheckTest.java
88826a0 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/ServicesYarnWorkPreservingCheckTest.java
98cfb18 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/UpgradeCheckOrderTest.java
7d70311 
>   ambari-server/src/test/java/org/apache/ambari/server/checks/YarnTimelineServerStatePreservingCheckTest.java
7469bbc 
> 
> Diff: https://reviews.apache.org/r/41569/diff/
> 
> 
> Testing
> -------
> 
> Verified it worked on the handful of tests I annotated.
> 
> 
> Thanks,
> 
> Alejandro Fernandez
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message