hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject scoping integration tests
Date Mon, 28 Nov 2011 20:23:43 GMT
The following discussion is closely related to HBASE-4712.

We should reach general consensus so that the execution of future test
strategy is smooth.

On Mon, Nov 28, 2011 at 11:50 AM, Jesse Yates <jesse.k.yates@gmail.com>wrote:

> I was considering 'integration tests' as a separate concern from the
> large/medium/small _unit_ tests.
>
> That is, in fact, why the failsafe plugin was added (and is designed for).
>
> Currently, we have a lot of tests that fall in the realm of integration
> tests (testing integration between various pieces, rather than single
> functionality alone). The most obvious indicator that something is an
> integration test is when it is using the MiniHBaseCluster since that is
> really testing the integration between the new feature and the rest of the
> system.
>
> The surefire plugin should really be testing unit tests, and then those
> can be classified as small/medium/large. However, the large would be
> _extremely_ rare cases.
>
> If the community is completely moving to just using the annotations for
> denoting size of tests, rather than using the integrationTest/unit test
> stuff, thats fine. However, I still think that if we do that, we should
> have a separate designation for @IntegrationTest (but that is really
> already covered in the fact that they would be named differently and
> therefore run by the different plugin). The reason we need to split them
> out is to have that separate tier of testing - once they pass unit level,
> they can be tested at the integration level and then on the acceptance test
> level (though we don't have any of those yet).
>
> This probably means more reclassification of tests.
>
> Making this explicit is important so people know at what point in the
> process their patch breaks (as well as staging the testing being good
> practice in general). Therefore, the small/medium/large classifications are
> not complete. Further, we should should still be able to use the general
> maven testing (mvn test, mvn verify) to run all the tests as expected,
> rather than having to know which things extra commands to run. To that end,
> we need to integrate HBASE-4712<https://issues.apache.org/jira/browse/HBASE-4712>ASAP
so its in the book and people know how to work with the current
> testing layout.
>
> I've been a bit behind on the testing discussions on dev@, so didn't get
> to put these thoughts in, but I think they can still be added on. This was
> also part of what Doug, Stack and I were kind of envisioning in the
> original email to the list.
>
> Thanks.
>
> -Jesse
>
>
> On Mon, Nov 28, 2011 at 11:29 AM, N Keywal <nkeywal@gmail.com> wrote:
>
>> Today's tests are:
>>  - 416 small tests, executed in ~3 minutes
>>  - 489 medium tests, executed in ~35 minutes (without parallelization)
>>  - 280 large tests, executed in ~90 minutes (without parallelization)
>>
>> So may be you could put this new test as large? Will you want to run this
>> test for all central builds? Or do you see it as a specific tests that will
>> be executed only sometimes?
>>
>>
>> On Mon, Nov 28, 2011 at 8:24 PM, N Keywal <nkeywal@gmail.com> wrote:
>>
>>> So I guess it's likely to be filtered as it's "not large".
>>>
>>>
>>> On Mon, Nov 28, 2011 at 8:17 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>
>>>> N:
>>>> Currently IntegrationTestConstraint doesn't have @Category annotation.
>>>>
>>>> FYI
>>>>
>>>>
>>>> On Mon, Nov 28, 2011 at 11:08 AM, N Keywal <nkeywal@gmail.com> wrote:
>>>>
>>>>> Yes, I put the large test as integration tests (may be 3 weeks ago?).
>>>>> So if you mix naming pattern with categories, you can end up with no
tests
>>>>> to do.
>>>>>
>>>>> It seems logical to me to put large tests in the failsafe module. If
>>>>> you want to use the failsafe tests for something else, I can remove then.
>>>>>
>>>>> see the doc in HBASE-4712<https://issues.apache.org/jira/browse/HBASE-4712>(last
comment) and the patch
>>>>> HBASE-4847 <https://issues.apache.org/jira/browse/HBASE-4847> on
how
>>>>> it designed to work today.
>>>>>
>>>>> Cheers,
>>>>>
>>>>> N.
>>>>>
>>>>>
>>>>> On Mon, Nov 28, 2011 at 8:03 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>>>
>>>>>> Copying N.
>>>>>>
>>>>>> On Mon, Nov 28, 2011 at 10:36 AM, Jesse Yates <
>>>>>> jesse.k.yates@gmail.com> wrote:
>>>>>>
>>>>>>> Hmmm, that's odd. It was working last week, maybe it was the
recent
>>>>>>> changes N put in.
>>>>>>>
>>>>>>> I'll look into it.
>>>>>>>
>>>>>>> Thanks!
>>>>>>>
>>>>>>> - Jesse Yates
>>>>>>>
>>>>>>> Sent from my iPhone.
>>>>>>>
>>>>>>> On Nov 28, 2011, at 9:51 AM, Ted Yu <yuzhihong@gmail.com>
wrote:
>>>>>>>
>>>>>>> > Hi,
>>>>>>> > Please categorize new tests for 4605.
>>>>>>> >
>>>>>>> > When I issued this command:
>>>>>>> > mvn failsafe:integration-test
>>>>>>> >
>>>>>>> > I got:
>>>>>>> >
>>>>>>> > Results :
>>>>>>> >
>>>>>>> > Tests run: 0, Failures: 0, Errors: 0, Skipped: 0
>>>>>>> >
>>>>>>> > Do you have some idea ?
>>>>>>> >
>>>>>>> > Thanks
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
>
> --
> -------------------
> Jesse Yates
> 240-888-2200
> @jesse_yates
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message