couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject The JavaScript Test Suite
Date Sun, 11 Oct 2015 10:00:19 GMT
(I need your help deciding what to do with a bunch of tests, jump to “Discussion” below
if you don’t care about the details before.)

* * *

Hi all,

I spent some more time getting the 1.x JavaScript tests running on the cluster port of 2.0.
WIP here: https://github.com/apache/couchdb/pull/354 — I’ve gotten a lot further, but
there is still a lot to do and everyone here can help, I’ve posted instructions on how to
do so further down. No Erlang knowledge needed, this is ideal for folks who want to start
with contributing to CouchDB.


Notable changes:

- instead of fixed test db names like `test_suite_db` that are used in almost every test,
we now generate a random database name for each test. The JS tests expected db deletion to
be synchronous, and that’s no longer the case in 2.0. Instead of error-prone polling to
wait for db deletion, we just use new db for every test. An added benefit now is that (some)
tests can run in parallel.

- tests now clean up after themselves (incomplete): tests previously only collectively used
three or four databases, and deleted and re-created them before each test was run. Now with
the change of creating a new database per test, we also need to clean up databases at the
end of a test, because otherwise we a) leave a lot of databases around and b) CouchDB runs
out of file descriptors during the test run. The cleanup isn’t 100% done yet, so we should
check for stray databases after JS test runs until we’ve got it cleaned. One particular
aspect here is when a test fails, it never reaches the cleanup code, so we might have to wrap
everything into a try/catch block.

- I disabled all tests that still fail and made them so they print the reason why they are
failing, so that fixing them should be a lot easier now. Some of them print just “TODO”,
their error cause is yet to be determined.

- couchdb.query() which previously executed temporary views, now under the hood creates a
design doc with a random name and queries that view. This fixes all tests relying on temp
views without having to change them all individually. This is because 2.0 doesn’t have temp
views anymore (which is a good thing).

- restartServer() is no longer used (mostly).

I already filed a few bugs that showed things missing in the cluster API that we’d expect
to be there:

- https://issues.apache.org/jira/browse/COUCHDB-2849
- https://issues.apache.org/jira/browse/COUCHDB-2850
- https://issues.apache.org/jira/browse/COUCHDB-2851

Based on this I am ever more convinced we should get this test suite work against the 2.0
clustered API, because we’ll be finding some more inconsistencies that we should fix before
we invite beta testers.


## Discussion:

We have a whole number of tests that use the test-internal function run_on_modified_server.
They use the _config API to test CouchDB in different configurations from the default one.
All of these tests now throw an error that _config isn’t available on the clustered API.
How should we handle them?

The tests are:

- erlang_views.js
- jsonp.js
- oauth_users_db.js
- oauth_users_db.js
- proxyauth.js
- reader_acl.js
- rewrite.js
- security_validation.js
- show_documents.js
- users_db.js
- view_errors.js


And a few tests aside from the proper compaction tests call _compact which is also not available
on the cluster. What to do with them?

- recreate_doc.js
- and a few more that are currently disabled for other reasons, sorry I don’t have a full
list at this point.

* * *


Bottom line: It’d be great if other folks here could join in the fun. No Erlang needed,
this is purely JavaScript and CouchDB HTTP API-work :)


## How to help:

1. Check out https://github.com/janl/couchdb/tree/js-tests-on-2.0-wip
2. run `make clean && ./configure -c --disable-docs --disable-fauxton && make`
3. `make javascript` runs the whole test suite

To run the whole test suite without waiting for rebar to confirm that no Erlang code needs
to be compiled, run `./test/javascript/run` for the whole test suite, or `./test/javascript/run
all_docs` to run an individual test (all_docs runs test/javascript/tests/all_docs.js, other
file names match accordingly).

Feel free to send me PRs against the branch on https://github.com/janl/couchdb/tree/js-tests-on-2.0-wip,
I’ll merge them, so the PR on apache/couchdb gets updated.

I’m looking forward to your contributions! :)

If you have any questions or have trouble getting started or just would like some hand-holding,
let me know!


Best
Jan
--




Mime
View raw message