hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: scoping integration tests
Date Wed, 30 Nov 2011 06:08:30 GMT
>From Stack:
-------
I think we can have it so everyone gets what they need;
small/medium/large are done by surefire and then we have a new
intergration test green field that is run by failsafe that can help us
get tests for bigtop et al. to run.

------

I think the above proposal makes sense.

Please share your comments.

On Mon, Nov 28, 2011 at 12:23 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> The following discussion is closely related to HBASE-4712.
>
> We should reach general consensus so that the execution of future test
> strategy is smooth.
>
> On Mon, Nov 28, 2011 at 11:50 AM, Jesse Yates <jesse.k.yates@gmail.com>wrote:
>
>> I was considering 'integration tests' as a separate concern from the
>> large/medium/small _unit_ tests.
>>
>> That is, in fact, why the failsafe plugin was added (and is designed
>> for).
>>
>> Currently, we have a lot of tests that fall in the realm of integration
>> tests (testing integration between various pieces, rather than single
>> functionality alone). The most obvious indicator that something is an
>> integration test is when it is using the MiniHBaseCluster since that is
>> really testing the integration between the new feature and the rest of the
>> system.
>>
>> The surefire plugin should really be testing unit tests, and then those
>> can be classified as small/medium/large. However, the large would be
>> _extremely_ rare cases.
>>
>> If the community is completely moving to just using the annotations for
>> denoting size of tests, rather than using the integrationTest/unit test
>> stuff, thats fine. However, I still think that if we do that, we should
>> have a separate designation for @IntegrationTest (but that is really
>> already covered in the fact that they would be named differently and
>> therefore run by the different plugin). The reason we need to split them
>> out is to have that separate tier of testing - once they pass unit level,
>> they can be tested at the integration level and then on the acceptance test
>> level (though we don't have any of those yet).
>>
>> This probably means more reclassification of tests.
>>
>> Making this explicit is important so people know at what point in the
>> process their patch breaks (as well as staging the testing being good
>> practice in general). Therefore, the small/medium/large classifications are
>> not complete. Further, we should should still be able to use the general
>> maven testing (mvn test, mvn verify) to run all the tests as expected,
>> rather than having to know which things extra commands to run. To that end,
>> we need to integrate HBASE-4712<https://issues.apache.org/jira/browse/HBASE-4712>ASAP
so its in the book and people know how to work with the current
>> testing layout.
>>
>> I've been a bit behind on the testing discussions on dev@, so didn't get
>> to put these thoughts in, but I think they can still be added on. This was
>> also part of what Doug, Stack and I were kind of envisioning in the
>> original email to the list.
>>
>> Thanks.
>>
>> -Jesse
>>
>>
>> On Mon, Nov 28, 2011 at 11:29 AM, N Keywal <nkeywal@gmail.com> wrote:
>>
>>> Today's tests are:
>>>  - 416 small tests, executed in ~3 minutes
>>>  - 489 medium tests, executed in ~35 minutes (without parallelization)
>>>  - 280 large tests, executed in ~90 minutes (without parallelization)
>>>
>>> So may be you could put this new test as large? Will you want to run
>>> this test for all central builds? Or do you see it as a specific tests that
>>> will be executed only sometimes?
>>>
>>>
>>> On Mon, Nov 28, 2011 at 8:24 PM, N Keywal <nkeywal@gmail.com> wrote:
>>>
>>>> So I guess it's likely to be filtered as it's "not large".
>>>>
>>>>
>>>> On Mon, Nov 28, 2011 at 8:17 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>>
>>>>> N:
>>>>> Currently IntegrationTestConstraint doesn't have @Category annotation.
>>>>>
>>>>> FYI
>>>>>
>>>>>
>>>>> On Mon, Nov 28, 2011 at 11:08 AM, N Keywal <nkeywal@gmail.com>
wrote:
>>>>>
>>>>>> Yes, I put the large test as integration tests (may be 3 weeks ago?).
>>>>>> So if you mix naming pattern with categories, you can end up with
no tests
>>>>>> to do.
>>>>>>
>>>>>> It seems logical to me to put large tests in the failsafe module.
If
>>>>>> you want to use the failsafe tests for something else, I can remove
then.
>>>>>>
>>>>>> see the doc in HBASE-4712<https://issues.apache.org/jira/browse/HBASE-4712>(last
comment) and the patch
>>>>>> HBASE-4847 <https://issues.apache.org/jira/browse/HBASE-4847>
on how
>>>>>> it designed to work today.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> N.
>>>>>>
>>>>>>
>>>>>> On Mon, Nov 28, 2011 at 8:03 PM, Ted Yu <yuzhihong@gmail.com>
wrote:
>>>>>>
>>>>>>> Copying N.
>>>>>>>
>>>>>>> On Mon, Nov 28, 2011 at 10:36 AM, Jesse Yates <
>>>>>>> jesse.k.yates@gmail.com> wrote:
>>>>>>>
>>>>>>>> Hmmm, that's odd. It was working last week, maybe it was
the recent
>>>>>>>> changes N put in.
>>>>>>>>
>>>>>>>> I'll look into it.
>>>>>>>>
>>>>>>>> Thanks!
>>>>>>>>
>>>>>>>> - Jesse Yates
>>>>>>>>
>>>>>>>> Sent from my iPhone.
>>>>>>>>
>>>>>>>> On Nov 28, 2011, at 9:51 AM, Ted Yu <yuzhihong@gmail.com>
wrote:
>>>>>>>>
>>>>>>>> > Hi,
>>>>>>>> > Please categorize new tests for 4605.
>>>>>>>> >
>>>>>>>> > When I issued this command:
>>>>>>>> > mvn failsafe:integration-test
>>>>>>>> >
>>>>>>>> > I got:
>>>>>>>> >
>>>>>>>> > Results :
>>>>>>>> >
>>>>>>>> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
>>>>>>>> >
>>>>>>>> > Do you have some idea ?
>>>>>>>> >
>>>>>>>> > Thanks
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>> --
>> -------------------
>> Jesse Yates
>> 240-888-2200
>> @jesse_yates
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message