httpd-test-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Geoffrey Young <ge...@modperlcookbook.org>
Subject Re: cvs commit: httpd-test/perl-framework/t/security CAN-2004-0940.t
Date Thu, 28 Oct 2004 19:51:52 GMT

> Did you change PHP version too? That's a PHP test, the result shouldn't
> change unless you change PHP version too with 1.3.33?

ah, I had php installed for 1.3.32 but not 1.3.33 :)

btw, the php stuff we've been doing is coming along quite well.  you might
be interested in the tarball chris and I are working on for apachecon

  http://www.modperlcookbook.org/~geoff/slides/nyphp/perl-php-test.tar.gz

note you'll need to have current A-T cvs installed, as I didn't take any
protection against missing versions, etc.

> Welll... we started having this debate a while back :)

indeed :)

> 
> Here's my take: I think it's correct to:
> 
> 1) only test for new features in versions on which they are known to be 
> present

I think we agreed on that (eventually :)

> 
> 2) test for bugs in all versions unconditionally in all affected
> versions
> 
> I think it's the desired outcome that if you test 1.3.32 for
> CAN-2004-0940, it should fail: 1.3.32 is after all vulnerable to
> CAN-2004-0940.  Why hide that by skipping the test?  Likewise, if you're
> running 1.3.32 you *should* be told that there is a nasty mod_rewrite
> regression in that version.
> 
> Maybe I'm hawking my corporate agenda here a little too, because it
> makes httpd-test slightly more useful to me since I can test for 1.3.x +
> backported patch, whereas if the test was skipped for <1.3.33 it won't
> demonstrate that the code is patched.
> 
> Does that make sense?

sure.

what it really feels like is that we (as a community) need a new function of
sorts.  that is, skip just glosses over a failure, and todo is only forward
looking (throwing unexpectedly succeeded warnings) - we need some kind of
'known issue' marker that understands an issue can never be fixed (unlike
todo which assumes that it can be fixed in the future).

but I guess that's another topic altogether :)

so I guess I'm inclined to agree with your logic then - it's better to have
regressions fail loudly and pique some interest than to just gloss over
them, especially for security-type things.

thanks for being patient with me while I caught up, then :)

--Geoff

Mime
View raw message