couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastian Rothbucher <sebastianrothbuc...@googlemail.com>
Subject Re: The JavaScript Test Suite
Date Sun, 11 Oct 2015 16:53:34 GMT
Hi Jan,

I transferred some of my findings to your branch - pls. find a PR for the
first few tests submitted 2 your branch. I put in TODOs where either there
is most probably a bug (filed COUCHDB-2852 and COUCHDB-2853) or we have to
do some more work (like making config available).

Will from the Pouch team had the great idea of using -n 1 to mitigate
Heisenbugs due to distribution. I adopted it and it did work quite well. It
might make sense to put this into the dev/run call as a general rule, esp.
when checking for 201 (VS 202) status codes. Probably you know more on
that.

Anyways: I think we have to keep using _config on the local port as many
tests (like auth tests) depend on changed configurations. BTW: w/ -n 1 it's
only one call, but calling 3 _config(s) migt also work.
Hopefully (I just arrived at some test w/ that recently) we can spare
ourselves the _compact thing. I think we'd have to compact every single
shard via local port, right? for the first tests, we can go w/out as it
seems. Maybe we should stick to that for a while.

Looking fw 2 your thoughts

Best
    Sebastian

P.S::  Maybe we can adopt some stuff from
https://github.com/apache/couchdb/compare/master...sebastianrothbucher:clustertest

P.P.S.: Will keep working on it, just not in the next few days,
unfortunately ;-(

On Sun, Oct 11, 2015 at 12:00 PM, Jan Lehnardt <jan@apache.org> wrote:

> (I need your help deciding what to do with a bunch of tests, jump to
> “Discussion” below if you don’t care about the details before.)
>
> * * *
>
> Hi all,
>
> I spent some more time getting the 1.x JavaScript tests running on the
> cluster port of 2.0. WIP here: https://github.com/apache/couchdb/pull/354
> — I’ve gotten a lot further, but there is still a lot to do and everyone
> here can help, I’ve posted instructions on how to do so further down. No
> Erlang knowledge needed, this is ideal for folks who want to start with
> contributing to CouchDB.
>
>
> Notable changes:
>
> - instead of fixed test db names like `test_suite_db` that are used in
> almost every test, we now generate a random database name for each test.
> The JS tests expected db deletion to be synchronous, and that’s no longer
> the case in 2.0. Instead of error-prone polling to wait for db deletion, we
> just use new db for every test. An added benefit now is that (some) tests
> can run in parallel.
>
> - tests now clean up after themselves (incomplete): tests previously only
> collectively used three or four databases, and deleted and re-created them
> before each test was run. Now with the change of creating a new database
> per test, we also need to clean up databases at the end of a test, because
> otherwise we a) leave a lot of databases around and b) CouchDB runs out of
> file descriptors during the test run. The cleanup isn’t 100% done yet, so
> we should check for stray databases after JS test runs until we’ve got it
> cleaned. One particular aspect here is when a test fails, it never reaches
> the cleanup code, so we might have to wrap everything into a try/catch
> block.
>
> - I disabled all tests that still fail and made them so they print the
> reason why they are failing, so that fixing them should be a lot easier
> now. Some of them print just “TODO”, their error cause is yet to be
> determined.
>
> - couchdb.query() which previously executed temporary views, now under the
> hood creates a design doc with a random name and queries that view. This
> fixes all tests relying on temp views without having to change them all
> individually. This is because 2.0 doesn’t have temp views anymore (which is
> a good thing).
>
> - restartServer() is no longer used (mostly).
>
> I already filed a few bugs that showed things missing in the cluster API
> that we’d expect to be there:
>
> - https://issues.apache.org/jira/browse/COUCHDB-2849
> - https://issues.apache.org/jira/browse/COUCHDB-2850
> - https://issues.apache.org/jira/browse/COUCHDB-2851
>
> Based on this I am ever more convinced we should get this test suite work
> against the 2.0 clustered API, because we’ll be finding some more
> inconsistencies that we should fix before we invite beta testers.
>
>
> ## Discussion:
>
> We have a whole number of tests that use the test-internal function
> run_on_modified_server. They use the _config API to test CouchDB in
> different configurations from the default one. All of these tests now throw
> an error that _config isn’t available on the clustered API. How should we
> handle them?
>
> The tests are:
>
> - erlang_views.js
> - jsonp.js
> - oauth_users_db.js
> - oauth_users_db.js
> - proxyauth.js
> - reader_acl.js
> - rewrite.js
> - security_validation.js
> - show_documents.js
> - users_db.js
> - view_errors.js
>
>
> And a few tests aside from the proper compaction tests call _compact which
> is also not available on the cluster. What to do with them?
>
> - recreate_doc.js
> - and a few more that are currently disabled for other reasons, sorry I
> don’t have a full list at this point.
>
> * * *
>
>
> Bottom line: It’d be great if other folks here could join in the fun. No
> Erlang needed, this is purely JavaScript and CouchDB HTTP API-work :)
>
>
> ## How to help:
>
> 1. Check out https://github.com/janl/couchdb/tree/js-tests-on-2.0-wip
> 2. run `make clean && ./configure -c --disable-docs --disable-fauxton &&
> make`
> 3. `make javascript` runs the whole test suite
>
> To run the whole test suite without waiting for rebar to confirm that no
> Erlang code needs to be compiled, run `./test/javascript/run` for the whole
> test suite, or `./test/javascript/run all_docs` to run an individual test
> (all_docs runs test/javascript/tests/all_docs.js, other file names match
> accordingly).
>
> Feel free to send me PRs against the branch on
> https://github.com/janl/couchdb/tree/js-tests-on-2.0-wip, I’ll merge
> them, so the PR on apache/couchdb gets updated.
>
> I’m looking forward to your contributions! :)
>
> If you have any questions or have trouble getting started or just would
> like some hand-holding, let me know!
>
>
> Best
> Jan
> --
>
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message