couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: The JavaScript Test Suite
Date Mon, 12 Oct 2015 08:20:23 GMT
Hi Andy,

thanks for giving this a spin!

> On 11 Oct 2015, at 22:51, Andy Wenk <andywenk@apache.org> wrote:
> 
> Hi Jan,
> 
> first of all thanks a lot for the great work. I followed your instructions
> and have some notes here.
> 
> * I think it should be mentioned, that one has to
> 
>  git checkout js-tests-on-2.0-wip
> 
> because when clicking the link in the email, you will be sent to the
> correct branch, but when you copy the clone URL on that page, it will be
> https://github.com/janl/couchdb.git.

Ah thanks, yeah, sorry, I hoped it was clear from the URL, thanks for making
this a little more explicit :)

> * make clean is only working, if one has already configured the repo once.
> So if not, simply use
> 
> ./configure -c --disable-docs --disable-fauxton && make

Hah thanks, or skip the step if you made a fresh clone :)

> * At the moment I let ./test/javascript/run . All tetst fail so far. Is
> that correct?

that is not correct, what do they fail with? Do you get any error messages?

> 
> More later.
> 
> All the best
> 
> Andy
> 
> On 11 October 2015 at 18:53, Sebastian Rothbucher <
> sebastianrothbucher@googlemail.com> wrote:
> 
>> Hi Jan,
>> 
>> I transferred some of my findings to your branch - pls. find a PR for the
>> first few tests submitted 2 your branch. I put in TODOs where either there
>> is most probably a bug (filed COUCHDB-2852 and COUCHDB-2853) or we have to
>> do some more work (like making config available).
>> 
>> Will from the Pouch team had the great idea of using -n 1 to mitigate
>> Heisenbugs due to distribution. I adopted it and it did work quite well. It
>> might make sense to put this into the dev/run call as a general rule, esp.
>> when checking for 201 (VS 202) status codes. Probably you know more on
>> that.
>> 
>> Anyways: I think we have to keep using _config on the local port as many
>> tests (like auth tests) depend on changed configurations. BTW: w/ -n 1 it's
>> only one call, but calling 3 _config(s) migt also work.
>> Hopefully (I just arrived at some test w/ that recently) we can spare
>> ourselves the _compact thing. I think we'd have to compact every single
>> shard via local port, right? for the first tests, we can go w/out as it
>> seems. Maybe we should stick to that for a while.
>> 
>> Looking fw 2 your thoughts
>> 
>> Best
>>    Sebastian
>> 
>> P.S::  Maybe we can adopt some stuff from
>> 
>> https://github.com/apache/couchdb/compare/master...sebastianrothbucher:clustertest
>> 
>> P.P.S.: Will keep working on it, just not in the next few days,
>> unfortunately ;-(
>> 
>> On Sun, Oct 11, 2015 at 12:00 PM, Jan Lehnardt <jan@apache.org> wrote:
>> 
>>> (I need your help deciding what to do with a bunch of tests, jump to
>>> “Discussion” below if you don’t care about the details before.)
>>> 
>>> * * *
>>> 
>>> Hi all,
>>> 
>>> I spent some more time getting the 1.x JavaScript tests running on the
>>> cluster port of 2.0. WIP here:
>> https://github.com/apache/couchdb/pull/354
>>> — I’ve gotten a lot further, but there is still a lot to do and everyone
>>> here can help, I’ve posted instructions on how to do so further down. No
>>> Erlang knowledge needed, this is ideal for folks who want to start with
>>> contributing to CouchDB.
>>> 
>>> 
>>> Notable changes:
>>> 
>>> - instead of fixed test db names like `test_suite_db` that are used in
>>> almost every test, we now generate a random database name for each test.
>>> The JS tests expected db deletion to be synchronous, and that’s no longer
>>> the case in 2.0. Instead of error-prone polling to wait for db deletion,
>> we
>>> just use new db for every test. An added benefit now is that (some) tests
>>> can run in parallel.
>>> 
>>> - tests now clean up after themselves (incomplete): tests previously only
>>> collectively used three or four databases, and deleted and re-created
>> them
>>> before each test was run. Now with the change of creating a new database
>>> per test, we also need to clean up databases at the end of a test,
>> because
>>> otherwise we a) leave a lot of databases around and b) CouchDB runs out
>> of
>>> file descriptors during the test run. The cleanup isn’t 100% done yet, so
>>> we should check for stray databases after JS test runs until we’ve got it
>>> cleaned. One particular aspect here is when a test fails, it never
>> reaches
>>> the cleanup code, so we might have to wrap everything into a try/catch
>>> block.
>>> 
>>> - I disabled all tests that still fail and made them so they print the
>>> reason why they are failing, so that fixing them should be a lot easier
>>> now. Some of them print just “TODO”, their error cause is yet to be
>>> determined.
>>> 
>>> - couchdb.query() which previously executed temporary views, now under
>> the
>>> hood creates a design doc with a random name and queries that view. This
>>> fixes all tests relying on temp views without having to change them all
>>> individually. This is because 2.0 doesn’t have temp views anymore (which
>> is
>>> a good thing).
>>> 
>>> - restartServer() is no longer used (mostly).
>>> 
>>> I already filed a few bugs that showed things missing in the cluster API
>>> that we’d expect to be there:
>>> 
>>> - https://issues.apache.org/jira/browse/COUCHDB-2849
>>> - https://issues.apache.org/jira/browse/COUCHDB-2850
>>> - https://issues.apache.org/jira/browse/COUCHDB-2851
>>> 
>>> Based on this I am ever more convinced we should get this test suite work
>>> against the 2.0 clustered API, because we’ll be finding some more
>>> inconsistencies that we should fix before we invite beta testers.
>>> 
>>> 
>>> ## Discussion:
>>> 
>>> We have a whole number of tests that use the test-internal function
>>> run_on_modified_server. They use the _config API to test CouchDB in
>>> different configurations from the default one. All of these tests now
>> throw
>>> an error that _config isn’t available on the clustered API. How should we
>>> handle them?
>>> 
>>> The tests are:
>>> 
>>> - erlang_views.js
>>> - jsonp.js
>>> - oauth_users_db.js
>>> - oauth_users_db.js
>>> - proxyauth.js
>>> - reader_acl.js
>>> - rewrite.js
>>> - security_validation.js
>>> - show_documents.js
>>> - users_db.js
>>> - view_errors.js
>>> 
>>> 
>>> And a few tests aside from the proper compaction tests call _compact
>> which
>>> is also not available on the cluster. What to do with them?
>>> 
>>> - recreate_doc.js
>>> - and a few more that are currently disabled for other reasons, sorry I
>>> don’t have a full list at this point.
>>> 
>>> * * *
>>> 
>>> 
>>> Bottom line: It’d be great if other folks here could join in the fun. No
>>> Erlang needed, this is purely JavaScript and CouchDB HTTP API-work :)
>>> 
>>> 
>>> ## How to help:
>>> 
>>> 1. Check out https://github.com/janl/couchdb/tree/js-tests-on-2.0-wip
>>> 2. run `make clean && ./configure -c --disable-docs --disable-fauxton
&&
>>> make`
>>> 3. `make javascript` runs the whole test suite
>>> 
>>> To run the whole test suite without waiting for rebar to confirm that no
>>> Erlang code needs to be compiled, run `./test/javascript/run` for the
>> whole
>>> test suite, or `./test/javascript/run all_docs` to run an individual test
>>> (all_docs runs test/javascript/tests/all_docs.js, other file names match
>>> accordingly).
>>> 
>>> Feel free to send me PRs against the branch on
>>> https://github.com/janl/couchdb/tree/js-tests-on-2.0-wip, I’ll merge
>>> them, so the PR on apache/couchdb gets updated.
>>> 
>>> I’m looking forward to your contributions! :)
>>> 
>>> If you have any questions or have trouble getting started or just would
>>> like some hand-holding, let me know!
>>> 
>>> 
>>> Best
>>> Jan
>>> --
>>> 
>>> 
>>> 
>>> 
>> 
> 
> 
> 
> -- 
> Andy Wenk
> Hamburg - Germany
> RockIt!
> 
> GPG fingerprint: C044 8322 9E12 1483 4FEC 9452 B65D 6BE3 9ED3 9588
> 
> https://people.apache.org/keys/committer/andywenk.asc

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/


Mime
View raw message