incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: Futon Test Suite
Date Thu, 11 Aug 2011 00:58:06 GMT
On Wed, Aug 10, 2011 at 12:49 AM, Paul Davis
<paul.joseph.davis@gmail.com> wrote:
> On Tue, Aug 9, 2011 at 8:11 PM, Filipe David Manana <fdmanana@apache.org> wrote:
>> On Mon, Aug 8, 2011 at 11:43 PM, Paul Davis <paul.joseph.davis@gmail.com> wrote:
>>> The entire test suite takes about 4 minutes to run on a semi recent
>>> MBA. Most of the tests are fairly speedy, but four of them stick out
>>> quite drastically:
>>>
>>> delayed_commits: 15s
>>> design_docs: 25s
>>> replication: 90s
>>> replcaitor_db: 60s
>>
>> The replication.js test grew a lot after the new replicator was
>> introduced. Basically it covers a lot more scenarios then the old
>> replication.js test, and tests with larger amounts of documents and
>> continuous replications.
>> I think this is a good thing and inevitable (due to bug fixes, new
>> features, etc).
>>
>> The replicator_db.js does several server restart calls, which are
>> necessary to test this feature.
>>
>> After Jan's patch to add a "verify installation" feature to Futon, I
>> don't think individual tests taking 1, 2 or 5 minutes are an issue, as
>> long as they succeed.
>> For a database management system, having much more comprehensive tests
>> (which mean that they take longer to run) is a good thing.
>>
>> I agree with everything said in this thread.
>>
>
> I only mention the replication tests specifically because it seems
> like they spend a lot of time polling database info objects and the
> logs fly by without any other log messages. I was mostly wondering if
> this was related to a gen_server timeout or commit_after message. On
> the other hand, we should also probably start thinking about
> hierarchical testing schemes. replication.js is over 1.5K loc which
> seems awfully heaving for a single test. These sorts of things will
> help when we want to run certain parts of the suite continuously while
> hacking and then run the full thing before committing.
>
> Also, I think you're spot on about Jan's patch. We should turn Futon's
> tests into a "Your node is functioning suite" and move the test suite
> to the CLI so we can be much more specific in our testing.
>
> And randomly it occurs to me that maybe we should re-evaluate our use
> of init:restart during testing. I know it gives us a clean slate, but
> perhaps having a "randomize test order" would be more useful for
> detecting failures that are non-obvious. Granted that introduces
> obvious difficulties with incompatible tests (ie things that test
> behavior for multiple values of a specific config setting).
>
>>>
>>> I haven't dug into these too much yet. The only thing I've noticed is
>>> that replication and relplicator_db seem to spend a lot of their time
>>> polling various URLs waiting for task completion. Perhaps I'm just
>>> being impatient but it seems that each poll lasts an uncessarily long
>>> time for a unit tests (3-5s) so I was wondering if we were hitting a
>>> timeout or something.
>>>
>>> If anyone wants to dig into the test suite and see if they can't speed
>>> these up or even just split them apart so we know what's taking awhile
>>> that'd be super awesome.
>>>
>>> Also, I've been thinking more and more about beefing up the JavaScript
>>> test suite runner and moving more of our browser tests over to
>>> dedicated code in those tests. If anyone's interested in hacking on
>>> some C and JavaScript against an HTTP API, let me know.
>>>
>>
>>
>>
>> --
>> Filipe David Manana,
>> fdmanana@gmail.com, fdmanana@apache.org
>>
>> "Reasonable men adapt themselves to the world.
>>  Unreasonable men adapt the world to themselves.
>>  That's why all progress depends on unreasonable men."
>>
>

Since no one seems to have believed me I decided to take a closer look
at replication.js tests. And as I pointed out it was just polling a
URL in a tight loop for 3s at a time. On my machine, this patch drops
replication.js from 93329ms to 41785ms. I'll point out that that's
more than twice as fast. And that was just an obvious optimization
from watching the log scroll. There are plenty more simple things that
could be done to speed these up.

Also, this patch makes me think that a _replication/localid -> JSON
status blob might be useful. Though I dunno how possible that is. I
reckon if we had that these would be sped up even more.


diff --git a/share/www/script/couch.js b/share/www/script/couch.js
index 304c9c1..792e638 100644
--- a/share/www/script/couch.js
+++ b/share/www/script/couch.js
@@ -40,6 +40,8 @@ function CouchDB(name, httpHeaders) {
     if (this.last_req.status == 404) {
       return false;
     }
+    var t0 = new Date();
+    while(true) {if((new Date()) - t0 > 100) break;}
     CouchDB.maybeThrowError(this.last_req);
     return JSON.parse(this.last_req.responseText);
   };
diff --git a/share/www/script/test/replication.js
b/share/www/script/test/replication.js
index 65c5eaa..b82375a 100644
--- a/share/www/script/test/replication.js
+++ b/share/www/script/test/replication.js
@@ -149,24 +149,40 @@ couchTests.replication = function(debug) {
   }


-  function waitForSeq(sourceDb, targetDb) {
-    var targetSeq,
-        sourceSeq = sourceDb.info().update_seq,
+  function waitForSeq(sourceDb, targetDb, rep_id) {
+    var seq = sourceDb.info().update_seq,
+        ri = new RegExp(rep_id),
+        tasks,
         t0 = new Date(),
         t1,
         ms = 3000;

     do {
-      targetSeq = targetDb.info().update_seq;
+      tasks = JSON.parse(CouchDB.request("GET",
"/_active_tasks").responseText);
+      for(var i = 0; i < tasks.length; i++) {
+        if(!ri.test(tasks[i].task)) continue;
+        var captured = /Processed (\d+)/.exec(tasks[i].status);
+        if(parseInt(captured[1]) >= seq) return;
+        break;
+      }
       t1 = new Date();
-    } while (((t1 - t0) <= ms) && targetSeq < sourceSeq);
+    } while ((t1 - t0) <= ms);
   }

+  function waitForRepEnd(rep_id) {
+    var ri = new RegExp(rep_id),
+        tasks,
+        t0 = new Date(),
+        t1,
+        ms = 3000;

-  function wait(ms) {
-    var t0 = new Date(), t1;
     do {
-      CouchDB.request("GET", "/");
+      tasks = JSON.parse(CouchDB.request("GET",
"/_active_tasks").responseText);
+      var found = false;
+      for(var i = 0; i < tasks.length; i++) {
+        if(!ri.test(tasks[i].task)) found = true;
+      }
+      if(!found) return;
       t1 = new Date();
     } while ((t1 - t0) <= ms);
   }
@@ -1143,7 +1159,7 @@ couchTests.replication = function(debug) {

     var rep_id = repResult._local_id;

-    waitForSeq(sourceDb, targetDb);
+    waitForSeq(sourceDb, targetDb, rep_id);

     for (j = 0; j < docs.length; j++) {
       doc = docs[j];
@@ -1181,7 +1197,7 @@ couchTests.replication = function(debug) {
     var ddoc = docs[docs.length - 1]; // design doc
     addAtt(sourceDb, ddoc, "readme.txt", att1_data, "text/plain");

-    waitForSeq(sourceDb, targetDb);
+    waitForSeq(sourceDb, targetDb, rep_id);

     var modifDocs = docs.slice(10, 15).concat([ddoc]);
     for (j = 0; j < modifDocs.length; j++) {
@@ -1226,7 +1242,7 @@ couchTests.replication = function(debug) {
     // add another attachment to the ddoc on source
     addAtt(sourceDb, ddoc, "data.dat", att2_data, "application/binary");

-    waitForSeq(sourceDb, targetDb);
+    waitForSeq(sourceDb, targetDb, rep_id);

     copy = targetDb.open(ddoc._id);
     var atts = copy._attachments;
@@ -1263,7 +1279,7 @@ couchTests.replication = function(debug) {
     var newDocs = makeDocs(25, 35);
     populateDb(sourceDb, newDocs, true);

-    waitForSeq(sourceDb, targetDb);
+    waitForSeq(sourceDb, targetDb, rep_id);

     for (j = 0; j < newDocs.length; j++) {
       doc = newDocs[j];
@@ -1282,7 +1298,7 @@ couchTests.replication = function(debug) {
     TEquals(true, sourceDb.deleteDoc(newDocs[0]).ok);
     TEquals(true, sourceDb.deleteDoc(newDocs[6]).ok);

-    waitForSeq(sourceDb, targetDb);
+    waitForSeq(sourceDb, targetDb, rep_id);

     copy = targetDb.open(newDocs[0]._id);
     TEquals(null, copy);
@@ -1317,7 +1333,7 @@ couchTests.replication = function(debug) {
     };
     TEquals(true, sourceDb.save(doc).ok);

-    wait(2000);
+    waitForRepEnd(rep_id);
     copy = targetDb.open(doc._id);
     TEquals(null, copy);
   }
@@ -1359,7 +1375,7 @@ couchTests.replication = function(debug) {

   var tasksAfter = JSON.parse(xhr.responseText);
   TEquals(tasks.length, tasksAfter.length);
-  waitForSeq(sourceDb, targetDb);
+  waitForSeq(sourceDb, targetDb, rep_id);
   T(sourceDb.open("30") !== null);

   // cancel replication

Mime
View raw message