Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
MIME-Version: 1.0
In-Reply-To: <51547C60.3000603@apache.org>
References: <51535F0B.7060408@apache.org>
	<51547C60.3000603@apache.org>
Date: Thu, 28 Mar 2013 17:48:42 +0000
Message-ID: 
 <CAPaJBx6S=WXL1zq9vWuuHJiQqbwueJuh4UW7KNHct-AvbU__HA@mail.gmail.com>
Subject: Re: Javascript Test Suite
From: Noah Slater <nslater@apache.org>
To: "dev@couchdb.apache.org" <dev@couchdb.apache.org>
Content-Type: multipart/alternative; boundary=14dae9341283f4ab3004d8ffc37a

--14dae9341283f4ab3004d8ffc37a
Content-Type: text/plain; charset=ISO-8859-1

Sounds excellent! Very happy about this work!


On 28 March 2013 17:22, Wendall Cada <wendallc@apache.org> wrote:

> On 03/27/2013 02:05 PM, Wendall Cada wrote:
>
>> In 1.3.0, there is a new part of the test suite to run the javascript
>> tests from the command line. I'm running into various issues on different
>> hardware/OS configurations. Mostly, tests hanging or timing out and
>> failing. These are really hard to troubleshoot, as they all pass just fine
>> if run individually.
>>
>> What I'm experimenting with today is rewriting how the tests are
>> implemented to be run one at a time from a loop in bash, versus a loop in
>> javascript. I think the failures I'm running into are improper
>> setup/teardown. There may be an issue with rapid delete and adding a db, or
>> rapidly starting and stopping couchdb, but I think this is not what's
>> happening in my failures.
>>
>> The nature of spidermonkey doesn't allow for spawning threads, or
>> sandboxing, etc, so it's hard looking at the test suite to see how I can
>> improve running all tests. I think it's far better to have the setup spawn
>> a new interpreter for each test. Tear down will kill the interpreter.
>>
>> Wendall
>>
> An update here. I spent some time with this yesterday.
>
> There are two problems here I'd like to address.
>
> 1. When running `make check`, the entire javascript test suite runs in a
> single interpreter thread. This means that if there are any broken tests,
> then entire suite bails, as it can crash the interpreter. As I said, to
> address this, I simply want to refactor the test suite to spawn a new
> interpreter thread for every test. This is working well for me with my
> quick work yesterday, and allows me to put a small delay in between tests
> to ensure proper tear down from the previous test.
>
> 2. Some of the tests are written in a way that can cause a race condition
> in the test itself. Quite a few places exist in the current javascript
> tests where infinite loops can get generated. I simply want to re-factor
> several of these tests to work as expected and avoid the race conditions
> entirely.
>
> One additional thought I had was to mimic the output of the etap tests a
> bit more. Show the entire path to the test file being run, with a status at
> the end of the line. Change the individual test run to include the full
> path to the test to be run. Additionally, the run scripts would benefit
> from having some help output. I'm in the process of gathering notes for the
> documentation as well.
>
> My thinking is that if the test suites are similar, it will be helpful to
> new developers in troubleshooting issues.
>
> If nobody has any objections, I'd like to get started on this work. I have
> a local branch in progress for this already, but it's mostly just a scratch
> space for testing so far.
>
> Wendall
>


-- 
NS

--14dae9341283f4ab3004d8ffc37a--