stdcxx-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Sebor <>
Subject Re: [jira] Updated: (STDCXX-536) allow thread safety tests to time out without failing
Date Wed, 12 Sep 2007 00:28:25 GMT
Travis Vitek wrote:
>> I guess I hadn't thought of the fact that some of these tests
>> create multiple thread pools. Hmm. I don't think the test should
>> be allowed to reset its own timeout but I'm not sure I see a good
>> solution.
> Yeah, ugh.
>> It certainly shouldn't exit successfully without doing
>> the rest of the work.
> Agreed.
>> Should it fail?
> Well, if we make the test fail aren't we essentially setting another
> hard limit?

You're right, we would be for all the tests that create multiple
thread pools.

>> Or should it just continue with the next pool and allow
>> itself to be killed?
> If we do that, then this timeout idea is done. 
> If the goal is to get the tests to run to completion sometime before the
> hard limit there are a few obvious changes we could make. In some cases
> we might be able to just reduce the default number of iterations.

The problem is that we don't know apriori how many iterations
each test will be able to complete in the specified amount of
[wall clock] time because the number depends on the hardware,
and on the system load at the time the test runs, in addition
to the test itself.

> In
> other cases, we could combine 'sections' of the test. Some tests have a
> section to test a char specialization, then a wchar_t specialization
> section, and finally a combined section that tests both. We could
> potentially remove the first two sections, or at least disable them by
> default.

Yes, that's a possibility.

> Another option would be to continue testing these sections, but
> separate them out into their own tests.

Right. That's the solution I keep ending up with as the most robust
one. The main advantage I see is minimizing the scope of the tested
functionality. The biggest disadvantage is the significant overhead
of building and running twice as many (thread safety) tests as we
have now. We could eliminate some of the overhead due to building
these tests by building just one binary and rigging our test driver
to invoke it two or more times with different options (I like this
feature in and of itself a lot) but I fear there is little we can
do to deal with the overhead of running the test(s) multiple times.

Over to you.


View raw message