couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: The JavaScript Test Suite
Date Mon, 12 Oct 2015 11:24:58 GMT
Ah, it looks I suck at instructions:

running make javascript starts the test cluster for you, with just ./test/javascript/run you
have to run dev/run in another terminal first.

Best
Jan
--

> On 12 Oct 2015, at 13:19, Andy Wenk <andywenk@apache.org> wrote:
> 
> Hey Jan,
> 
> I call it like this:
> 
> ~/p/c/couchdb-js-tests git:js-tests-on-2.0-wip: ./test/javascript/run
> test/javascript/tests/all_docs.js
> FAIL isRunning
> fail
> test/javascript/tests/attachment_names.js
> FAIL isRunning
> fail
> test/javascript/tests/attachment_paths.js
> FAIL isRunning
> fail
> test/javascript/tests/attachment_ranges.js
> FAIL isRunning
> fail
> test/javascript/tests/attachment_views.js
> FAIL isRunning
> fail
> test/javascript/tests/attachments.js
> FAIL isRunning
> fail
> test/javascript/tests/attachments_multipart.js
> FAIL isRunning
> fail
> 
> As you can see, the tests fail. And this is for all of them ;-) 
> 
> Am I missing sth? Btw. what says "FAIL isRunning"?
> 
> Cheers
> 
> Andy
> 
> On 12 October 2015 at 10:20, Jan Lehnardt <jan@apache.org> wrote:
> Hi Andy,
> 
> thanks for giving this a spin!
> 
> > On 11 Oct 2015, at 22:51, Andy Wenk <andywenk@apache.org> wrote:
> >
> > Hi Jan,
> >
> > first of all thanks a lot for the great work. I followed your instructions
> > and have some notes here.
> >
> > * I think it should be mentioned, that one has to
> >
> >  git checkout js-tests-on-2.0-wip
> >
> > because when clicking the link in the email, you will be sent to the
> > correct branch, but when you copy the clone URL on that page, it will be
> > https://github.com/janl/couchdb.git.
> 
> Ah thanks, yeah, sorry, I hoped it was clear from the URL, thanks for making
> this a little more explicit :)
> 
> > * make clean is only working, if one has already configured the repo once.
> > So if not, simply use
> >
> > ./configure -c --disable-docs --disable-fauxton && make
> 
> Hah thanks, or skip the step if you made a fresh clone :)
> 
> > * At the moment I let ./test/javascript/run . All tetst fail so far. Is
> > that correct?
> 
> that is not correct, what do they fail with? Do you get any error messages?
> 
> >
> > More later.
> >
> > All the best
> >
> > Andy
> >
> > On 11 October 2015 at 18:53, Sebastian Rothbucher <
> > sebastianrothbucher@googlemail.com> wrote:
> >
> >> Hi Jan,
> >>
> >> I transferred some of my findings to your branch - pls. find a PR for the
> >> first few tests submitted 2 your branch. I put in TODOs where either there
> >> is most probably a bug (filed COUCHDB-2852 and COUCHDB-2853) or we have to
> >> do some more work (like making config available).
> >>
> >> Will from the Pouch team had the great idea of using -n 1 to mitigate
> >> Heisenbugs due to distribution. I adopted it and it did work quite well. It
> >> might make sense to put this into the dev/run call as a general rule, esp.
> >> when checking for 201 (VS 202) status codes. Probably you know more on
> >> that.
> >>
> >> Anyways: I think we have to keep using _config on the local port as many
> >> tests (like auth tests) depend on changed configurations. BTW: w/ -n 1 it's
> >> only one call, but calling 3 _config(s) migt also work.
> >> Hopefully (I just arrived at some test w/ that recently) we can spare
> >> ourselves the _compact thing. I think we'd have to compact every single
> >> shard via local port, right? for the first tests, we can go w/out as it
> >> seems. Maybe we should stick to that for a while.
> >>
> >> Looking fw 2 your thoughts
> >>
> >> Best
> >>    Sebastian
> >>
> >> P.S::  Maybe we can adopt some stuff from
> >>
> >> https://github.com/apache/couchdb/compare/master...sebastianrothbucher:clustertest
> >>
> >> P.P.S.: Will keep working on it, just not in the next few days,
> >> unfortunately ;-(
> >>
> >> On Sun, Oct 11, 2015 at 12:00 PM, Jan Lehnardt <jan@apache.org> wrote:
> >>
> >>> (I need your help deciding what to do with a bunch of tests, jump to
> >>> “Discussion” below if you don’t care about the details before.)
> >>>
> >>> * * *
> >>>
> >>> Hi all,
> >>>
> >>> I spent some more time getting the 1.x JavaScript tests running on the
> >>> cluster port of 2.0. WIP here:
> >> https://github.com/apache/couchdb/pull/354
> >>> — I’ve gotten a lot further, but there is still a lot to do and everyone
> >>> here can help, I’ve posted instructions on how to do so further down.
No
> >>> Erlang knowledge needed, this is ideal for folks who want to start with
> >>> contributing to CouchDB.
> >>>
> >>>
> >>> Notable changes:
> >>>
> >>> - instead of fixed test db names like `test_suite_db` that are used in
> >>> almost every test, we now generate a random database name for each test.
> >>> The JS tests expected db deletion to be synchronous, and that’s no longer
> >>> the case in 2.0. Instead of error-prone polling to wait for db deletion,
> >> we
> >>> just use new db for every test. An added benefit now is that (some) tests
> >>> can run in parallel.
> >>>
> >>> - tests now clean up after themselves (incomplete): tests previously only
> >>> collectively used three or four databases, and deleted and re-created
> >> them
> >>> before each test was run. Now with the change of creating a new database
> >>> per test, we also need to clean up databases at the end of a test,
> >> because
> >>> otherwise we a) leave a lot of databases around and b) CouchDB runs out
> >> of
> >>> file descriptors during the test run. The cleanup isn’t 100% done yet,
so
> >>> we should check for stray databases after JS test runs until we’ve got
it
> >>> cleaned. One particular aspect here is when a test fails, it never
> >> reaches
> >>> the cleanup code, so we might have to wrap everything into a try/catch
> >>> block.
> >>>
> >>> - I disabled all tests that still fail and made them so they print the
> >>> reason why they are failing, so that fixing them should be a lot easier
> >>> now. Some of them print just “TODO”, their error cause is yet to be
> >>> determined.
> >>>
> >>> - couchdb.query() which previously executed temporary views, now under
> >> the
> >>> hood creates a design doc with a random name and queries that view. This
> >>> fixes all tests relying on temp views without having to change them all
> >>> individually. This is because 2.0 doesn’t have temp views anymore (which
> >> is
> >>> a good thing).
> >>>
> >>> - restartServer() is no longer used (mostly).
> >>>
> >>> I already filed a few bugs that showed things missing in the cluster API
> >>> that we’d expect to be there:
> >>>
> >>> - https://issues.apache.org/jira/browse/COUCHDB-2849
> >>> - https://issues.apache.org/jira/browse/COUCHDB-2850
> >>> - https://issues.apache.org/jira/browse/COUCHDB-2851
> >>>
> >>> Based on this I am ever more convinced we should get this test suite work
> >>> against the 2.0 clustered API, because we’ll be finding some more
> >>> inconsistencies that we should fix before we invite beta testers.
> >>>
> >>>
> >>> ## Discussion:
> >>>
> >>> We have a whole number of tests that use the test-internal function
> >>> run_on_modified_server. They use the _config API to test CouchDB in
> >>> different configurations from the default one. All of these tests now
> >> throw
> >>> an error that _config isn’t available on the clustered API. How should
we
> >>> handle them?
> >>>
> >>> The tests are:
> >>>
> >>> - erlang_views.js
> >>> - jsonp.js
> >>> - oauth_users_db.js
> >>> - oauth_users_db.js
> >>> - proxyauth.js
> >>> - reader_acl.js
> >>> - rewrite.js
> >>> - security_validation.js
> >>> - show_documents.js
> >>> - users_db.js
> >>> - view_errors.js
> >>>
> >>>
> >>> And a few tests aside from the proper compaction tests call _compact
> >> which
> >>> is also not available on the cluster. What to do with them?
> >>>
> >>> - recreate_doc.js
> >>> - and a few more that are currently disabled for other reasons, sorry I
> >>> don’t have a full list at this point.
> >>>
> >>> * * *
> >>>
> >>>
> >>> Bottom line: It’d be great if other folks here could join in the fun.
No
> >>> Erlang needed, this is purely JavaScript and CouchDB HTTP API-work :)
> >>>
> >>>
> >>> ## How to help:
> >>>
> >>> 1. Check out https://github.com/janl/couchdb/tree/js-tests-on-2.0-wip
> >>> 2. run `make clean && ./configure -c --disable-docs --disable-fauxton
&&
> >>> make`
> >>> 3. `make javascript` runs the whole test suite
> >>>
> >>> To run the whole test suite without waiting for rebar to confirm that no
> >>> Erlang code needs to be compiled, run `./test/javascript/run` for the
> >> whole
> >>> test suite, or `./test/javascript/run all_docs` to run an individual test
> >>> (all_docs runs test/javascript/tests/all_docs.js, other file names match
> >>> accordingly).
> >>>
> >>> Feel free to send me PRs against the branch on
> >>> https://github.com/janl/couchdb/tree/js-tests-on-2.0-wip, I’ll merge
> >>> them, so the PR on apache/couchdb gets updated.
> >>>
> >>> I’m looking forward to your contributions! :)
> >>>
> >>> If you have any questions or have trouble getting started or just would
> >>> like some hand-holding, let me know!
> >>>
> >>>
> >>> Best
> >>> Jan
> >>> --
> >>>
> >>>
> >>>
> >>>
> >>
> >
> >
> >
> > --
> > Andy Wenk
> > Hamburg - Germany
> > RockIt!
> >
> > GPG fingerprint: C044 8322 9E12 1483 4FEC 9452 B65D 6BE3 9ED3 9588
> >
> > https://people.apache.org/keys/committer/andywenk.asc
> 
> --
> Professional Support for Apache CouchDB:
> http://www.neighbourhood.ie/couchdb-support/
> 
> 
> 
> 
> -- 
> Andy Wenk
> Hamburg - Germany
> RockIt!
> 
> GPG fingerprint: C044 8322 9E12 1483 4FEC 9452 B65D 6BE3 9ED3 9588
> 
> https://people.apache.org/keys/committer/andywenk.asc

-- 
Professional Support for Apache CouchDB:
http://www.neighbourhood.ie/couchdb-support/


Mime
View raw message