stdcxx-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Sebor <>
Subject Re: [PATCH] Update test to validate results
Date Sun, 12 Aug 2007 21:39:08 GMT
Travis Vitek wrote:
>> -----Original Message-----
>> From: Martin Sebor [] 
>> Sent: Friday, August 10, 2007 1:42 PM
>> To:
>> Subject: Re: [PATCH] Update test to 
>> validate results
>> Travis Vitek wrote:
>>> Martin Sebor wrote:
>>>> We can't be using rw_assert() in a thread function (the driver
>>>> isn't thread safe yet, and if it was it would unnecessarily
>>>> synchronize the threads). Use the RW_ASSERT() macro instead.
>>> I think there will be a lot of useful context that is lost if
>>> I switch to RW_ASSERT. It would be especially useful to know
>>> the active locale, the input value, and the output values in
>>> these test cases.
>> These values are invaluable when the tests fail due to a problem
>> other than a thread safety bug in the library but I'm not sure
>> they would be of much use in the case of a thread safety bug
>> that manifests itself by corrupting data. Do you have specific
>> scenario in mind where this information would be useful?
> One example that I ran into in my limited testing was that a locale
> was getting created with the correct name, but it was not the correct
> locale that was getting used. The data wasn't corrupt, it was just
> for the wrong locale.
>   # INFO (S1) (3 lines):
>   # TEXT: testing std::time_put<charT> with 4 threads, 100000 iterations
> each, in locales { "en_US" "es_MX" }
>   # CLAUSE: lib.locale.time.put
>   # WARNING (S5) (3 lines):
>   # TEXT: locale="en_US" output="jue" expected="Thu"
>   # CLAUSE: lib.locale.time.put
> Sure this is a threading bug, but IMO the generated message is more
> useful than the other option

I agree that it looks more informative, but I'm not sure it really
is. Could you reproduce this consistently from run to run? If not,
it might be just a coincidence.

In any event, I agree that if we can avoid the synchronization in
the successful cases providing additional detail in the diagnostic
messages can be helpful. But rather than complicating the test
driver API with this detail I'd make the driver (i.e., calls to
rw_assert() et al) thread-safe. It should be straightforward to
do: simply add a global mutex to the driver and lock in on entry
to each of these functions.

> The other nice thing is that the test will continue, possibly exposing
> additional problems. 

I don't think we should trust the test after the first failure.
Also, since each thread function iterates thousands of times we
might end up with tens of thousands of lines of output. I don't
have a big problem with making this an option of the test just
so long as it's disabled by default (for nightly builds).

> Yes, that would be another workable solution. Honestly I'm hoping that
> once these tests are up and running the bugs will be found and there
> won't be any problems to speak of. If that is the case, then all of this
> discussion is moot.

Well, of course :) Once all the bugs have been fixed there will
be no failures to report. But the value of the tests will not only
be in helping us find and fix the existing bugs but also in helping
us catch future regressions.


View raw message