Mailing-List: contact dev-help@httpd.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@httpd.apache.org
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 9.2 \(3112\))
Subject: Re: collaboration request - apache server automation testing with
 swat tool
From: Jim Jagielski <jim@jaguNET.com>
In-Reply-To: 
 <CAL7UUk9Mc=MUVaQXPvc+mcaoik+=pFW7EW_ccTXbBmo9W0JB1Q@mail.gmail.com>
Date: Fri, 5 Feb 2016 09:01:41 -0500
Content-Transfer-Encoding: quoted-printable
Message-Id: <3A242214-802F-4DAE-A124-7FEB0B23D54D@jaguNET.com>
References: 
 <CAL7UUk-qyES+aMMSN6aBGi1rb=K3Q+pif4FbU__Lqkku7f2t0w@mail.gmail.com>
 <CACsi253b6=VufVyRW3J0iXYhMGRUzLYY1+uFRENAXfkD+Reaag@mail.gmail.com>
 <CAL7UUk_z-CgL1255w8ScgUW+oFtpuybvzoK_kQykbQNv025N_g@mail.gmail.com>
 <CACsi251twJxjTjEbjEnFuE2x49KX-BrVi=MSAs3-7L=ypaKQ7Q@mail.gmail.com>
 <CAL7UUk_1rvLqgNsOMKN-1GxeBWXb2Q3XVAjbbn4+1th9GdAOcw@mail.gmail.com>
 <CAL7UUk9Mc=MUVaQXPvc+mcaoik+=pFW7EW_ccTXbBmo9W0JB1Q@mail.gmail.com>
To: dev@httpd.apache.org

Personally, I like the idea of having another framework;
the current one is OK but somewhat "painful" to update.

I wonder how possible it would be to transcode the old tests
to Swat? We could then provide for 2 testing frameworks, one
developed by the ASF and the other external and 3rd party.


> On Feb 1, 2016, at 5:23 AM, Alexey Melezhik <melezhik@gmail.com> =
wrote:
>=20
> Here the list of existed issues I was able to automate tests for:
>=20
> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ ls -1 | grep -P =
'\d'
> 44221
> 46751
> 58789
> 58828
> 58854
>=20
>=20
> I have informed developers at https://bz.apache.org/bugzilla/ about
> these, so they rely upon them. Or you have some pre-release testing,
> so run all of these in a whole chunk?
>=20
> Regards
>=20
> Alexey
>=20
> PS chaining issues / tests listing  as always -
> https://github.com/melezhik/apache-swat
>=20
> PS2 it'd be could to verify tests as well against "bleeding edge"
> apache version gets installed from SCM, but I don't know how to do
> this.
> Currently I use http://httpd.apache.org/download.cgi ( last stable =
release )
>=20
>=20
>=20
> 2016-01-30 21:36 GMT+03:00 Alexey Melezhik <melezhik@gmail.com>:
>> Hi Bill!
>>=20
>> I have started to assemble swat tests  for apache2 issues coming from
>> https://bz.apache.org/bugzilla
>>=20
>> And this the first one - =
https://bz.apache.org/bugzilla/show_bug.cgi?id=3D44221 ,
>>=20
>> Test suite is failed for the moment :
>>=20
>>=20
>> vagrant@Debian-jessie-amd64-netboot:~/my/apache-swat$ swat -t 44221
>> /home/vagrant/.swat/.cache/384/prove/44221/foo.baz/00.GET.t ...
>> ok 1 - GET 127.0.0.1/44221/foo.baz succeeded
>> # http headers saved to =
/home/vagrant/.swat/.cache/384/prove/52jLAo76YZ.hdr
>> # body saved to /home/vagrant/.swat/.cache/384/prove/52jLAo76YZ
>> ok 2 - output match '200 OK'
>> 1..2
>> ok
>> /home/vagrant/.swat/.cache/384/prove/44221/FOO.bar/00.GET.t ...
>> ok 1 - GET 127.0.0.1/44221/FOO.bar succeeded
>> # http headers saved to =
/home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe.hdr
>> # body saved to /home/vagrant/.swat/.cache/384/prove/DvEZnO6ALe
>> ok 2 - output match '200 OK'
>> ok 3 - output match /Location: \S+/
>> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
>> 1..4
>> ok
>> /home/vagrant/.swat/.cache/384/prove/44221/foo.BAR/00.GET.t ...
>> ok 1 - GET 127.0.0.1/44221/foo.BAR succeeded
>> # http headers saved to =
/home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ.hdr
>> # body saved to /home/vagrant/.swat/.cache/384/prove/bWQ1Sbl0YZ
>> ok 2 - output match '200 OK'
>> ok 3 - output match /Location: \S+/
>> ok 4 - 'Location: http://127.0.0.1/44221/foo.bar' match 'foo.bar'
>> 1..4
>> ok
>> /home/vagrant/.swat/.cache/384/prove/44221/foo.bar/00.GET.t ...
>> ok 1 - GET 127.0.0.1/44221/foo.bar succeeded
>> # http headers saved to =
/home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv.hdr
>> # body saved to /home/vagrant/.swat/.cache/384/prove/Jd_hlo7Qwv
>> ok 2 - output match '200 OK'
>> 1..2
>> ok
>> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t ..
>> ok 1 - GET 127.0.0.1/44221/foo.html succeeded
>> # http headers saved to =
/home/vagrant/.swat/.cache/384/prove/tW2L511eym.hdr
>> # body saved to /home/vagrant/.swat/.cache/384/prove/tW2L511eym
>> ok 2 - output match /HTTP\/(\S+) (\d+) \S+/
>> not ok 3 - 'HTTP/1.1 300 Multiple Choices' match /404 /
>>=20
>> #   Failed test ''HTTP/1.1 300 Multiple Choices' match /404 /'
>> #   at /usr/local/share/perl/5.20.2/swat.pm line 218.
>> not ok 4 - output match 'Not Found'
>>=20
>> #   Failed test 'output match 'Not Found''
>> #   at /usr/local/share/perl/5.20.2/swat.pm line 218.
>> 1..4
>> # Looks like you failed 2 tests of 4.
>> Dubious, test returned 2 (wstat 512, 0x200)
>> Failed 2/4 subtests
>>=20
>> Test Summary Report
>> -------------------
>> /home/vagrant/.swat/.cache/384/prove/44221/foo.html/00.GET.t (Wstat:
>> 512 Tests: 4 Failed: 2)
>>  Failed tests:  3-4
>>  Non-zero exit status: 2
>> Files=3D5, Tests=3D16,  0 wallclock secs ( 0.03 usr  0.00 sys +  0.27 =
cusr
>> 0.00 csys =3D  0.30 CPU)
>> Result: FAIL
>>=20
>> ...
>>=20
>> Ok, this is just the beginning ;-))) , watch soon updates at -
>> https://github.com/melezhik/apache-swat
>>=20
>> 2016-01-29 20:44 GMT+03:00 William A Rowe Jr <wrowe@rowe-clan.net>:
>>> On Fri, Jan 29, 2016 at 6:35 AM, Alexey Melezhik =
<melezhik@gmail.com> wrote:
>>>>=20
>>>> Hi Bill!
>>>>=20
>>>> Any news? ( Please see my previous reply ...)
>>>> Intr
>>>=20
>>>=20
>>> Intrigued :)  But my responses will be delayed, I personally
>>> won't have time to look further myself until other backlogged
>>> commitments to httpd are caught up some more.
>>>=20
>>> Be aware many of us have only a few hours to participate in the
>>> httpd project itself, so it will take several days or more to gather
>>> feedback on your interest and perhaps a champion to drive such
>>> an effort if it is actively driven from this side.  Of course that
>>> should not stop you from assembling some ideas at the vcs of
>>> your choice, and sharing a link for those who want to look at an
>>> early draft of such an effort!
>>>=20
>>> Cheers,
>>>=20
>>> Bill