couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jan Lehnardt <...@apache.org>
Subject Re: Javascript Test Suite
Date Thu, 28 Mar 2013 18:21:57 GMT
+1s all around, this is much needed attention!

Feel free to file JIRAs for individual issues/milestones and discuss details there.

Jan
--

On Mar 28, 2013, at 18:22 , Wendall Cada <wendallc@apache.org> wrote:

> On 03/27/2013 02:05 PM, Wendall Cada wrote:
>> In 1.3.0, there is a new part of the test suite to run the javascript tests from
the command line. I'm running into various issues on different hardware/OS configurations.
Mostly, tests hanging or timing out and failing. These are really hard to troubleshoot, as
they all pass just fine if run individually.
>> 
>> What I'm experimenting with today is rewriting how the tests are implemented to be
run one at a time from a loop in bash, versus a loop in javascript. I think the failures I'm
running into are improper setup/teardown. There may be an issue with rapid delete and adding
a db, or rapidly starting and stopping couchdb, but I think this is not what's happening in
my failures.
>> 
>> The nature of spidermonkey doesn't allow for spawning threads, or sandboxing, etc,
so it's hard looking at the test suite to see how I can improve running all tests. I think
it's far better to have the setup spawn a new interpreter for each test. Tear down will kill
the interpreter.
>> 
>> Wendall
> An update here. I spent some time with this yesterday.
> 
> There are two problems here I'd like to address.
> 
> 1. When running `make check`, the entire javascript test suite runs in a single interpreter
thread. This means that if there are any broken tests, then entire suite bails, as it can
crash the interpreter. As I said, to address this, I simply want to refactor the test suite
to spawn a new interpreter thread for every test. This is working well for me with my quick
work yesterday, and allows me to put a small delay in between tests to ensure proper tear
down from the previous test.
> 
> 2. Some of the tests are written in a way that can cause a race condition in the test
itself. Quite a few places exist in the current javascript tests where infinite loops can
get generated. I simply want to re-factor several of these tests to work as expected and avoid
the race conditions entirely.
> 
> One additional thought I had was to mimic the output of the etap tests a bit more. Show
the entire path to the test file being run, with a status at the end of the line. Change the
individual test run to include the full path to the test to be run. Additionally, the run
scripts would benefit from having some help output. I'm in the process of gathering notes
for the documentation as well.
> 
> My thinking is that if the test suites are similar, it will be helpful to new developers
in troubleshooting issues.
> 
> If nobody has any objections, I'd like to get started on this work. I have a local branch
in progress for this already, but it's mostly just a scratch space for testing so far.
> 
> Wendall


Mime
View raw message