Mailing-List: contact lucy-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: lucy-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Date: Fri, 13 May 2011 11:17:44 -0700
From: Marvin Humphrey <marvin@rectangular.com>
To: lucy-dev@incubator.apache.org
Message-ID: <20110513181744.GA26737@rectangular.com>
References: <20110513000208.GA11742@rectangular.com>
 <935255.85330.qm@web161409.mail.bf1.yahoo.com>
 <20110513043306.GA12116@rectangular.com>
 <106510.18387.qm@web161407.mail.bf1.yahoo.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <106510.18387.qm@web161407.mail.bf1.yahoo.com>
User-Agent: Mutt/1.5.18 (2008-05-17)
Subject: Re: [lucy-dev] C TAP test harnesses

On Thu, May 12, 2011 at 09:59:47PM -0700, Joe Schaefer wrote:
> FWIW the features in the apreq stuff that are nice for me are
> 
> 1) macros provide access to __FILE__ and __LINE__ in the test source file,

It's definitely nice having access to __FILE__ and __LINE__.  The Charmonizer
tests don't include them because we prioritized taking variadic arguments for
the test label.  We couldn't keep that feature and include __FILE__ and
__LINE__ implicitly without requiring variadic macro support from the
compiler.

>From what I can tell, you made the opposite choice in the apreq harness -- you
support __FILE__ and __LINE__, but you only accept simple string test labels. 

If we were to change the Lucy test harness to use __FILE__ and __LINE__
implicitly, we would also have to modify some number of tests by creating
pre-formatted test labels locally to pass into the TEST_XXXXX() routines.  
A starter patch incorporating the necessary changes to TEST_TRUE() is below my
sig.  (It does not include fixes for the 22 incompatible variadic invocations
of TEST_TRUE().)

> 2) skip lists, todo lists, and fatals are supported.
> 
> 3) there is a localizer which allows you to next test frameworks.  it basically
> supplies its own harness and treats all subtest errors as fatal- handy if
> you're running a tight loop of tests and really aren't interested in feeding
> all that data directly to stdout.

These are great, too.   Our existing test harnesses have only a crude
mechanism for skipping tests, and no support for TODO tests or subtests.
 
> The apr dependency is just because apreq has a core dependency on apr, so why 
> not.

Whoa, if the apreq harness can work without APR that would be sweet.  I looked
at at.c and I see that it makes heavy use of apr_snprintf().  Can you fall
back to sprintf() safely?

> It could be dropped without much effort, or these features added to the
> existing Lucy C test apparatus.

Would apreq accept patches removing the APR dependency?  And is at.h a public
API?  Ideally, we would want to bundle those files and use them verbatim.
Maintaining our own fork for the sake of Charmonizer's tests is less
desirable, unless we decide we want to go all out and publish a supported C
TAP library ourselves.

Marvin Humphrey


diff --git a/core/Lucy/Test.c b/core/Lucy/Test.c
index 0457683..b0a69c2 100644
--- a/core/Lucy/Test.c
+++ b/core/Lucy/Test.c
@@ -56,13 +56,23 @@ TestBatch_plan(TestBatch *self) {
 }
 
 bool_t
-TestBatch_test_true(void *vself, bool_t condition, const char *pattern, ...) {
-    va_list args;
-    va_start(args, pattern);
-    bool_t result = TestBatch_VTest_True((TestBatch*)vself, condition,
-                                         pattern, args);
-    va_end(args);
-    return result;
+TestBatch_test_true(TestBatch *self, bool_t condition, const char *label,
+                    const char *file, int line) {
+    // Increment test number.
+    self->test_num++;
+
+    // Test condition and pass or fail.
+    if (condition) {
+        self->num_passed++;
+        printf("ok %" I64P " - %s\n", self->test_num, label);
+        return true;
+    }
+    else {
+        self->num_failed++;
+        printf("not ok %" I64P " - %s # (%s:%d)\n", self->test_num, label, 
+               file, line);
+        return false;
+    }
 }
 
 bool_t
@@ -135,29 +145,6 @@ TestBatch_skip(void *vself, const char *pattern, ...) {
 }
 
 bool_t
-TestBatch_vtest_true(TestBatch *self, bool_t condition, const char *pattern,
-                     va_list args) {
-    // Increment test number.
-    self->test_num++;
-
-    // Test condition and pass or fail.
-    if (condition) {
-        self->num_passed++;
-        printf("ok %" I64P " - ", self->test_num);
-        vprintf(pattern, args);
-        printf("\n");
-        return true;
-    }
-    else {
-        self->num_failed++;
-        printf("not ok %" I64P " - ", self->test_num);
-        vprintf(pattern, args);
-        printf("\n");
-        return false;
-    }
-}
-
-bool_t
 TestBatch_vtest_false(TestBatch *self, bool_t condition,
                       const char *pattern, va_list args) {
     // Increment test number.
diff --git a/core/Lucy/Test.cfh b/core/Lucy/Test.cfh
index 1b14224..91bec02 100644
--- a/core/Lucy/Test.cfh
+++ b/core/Lucy/Test.cfh
@@ -36,8 +36,9 @@ class Lucy::Test::TestBatch inherits Lucy::Object::Obj {
     void
     Plan(TestBatch *self);
 
-    inert bool_t
-    test_true(void *vself, bool_t condition, const char *pattern, ...);
+    bool_t
+    Test_True(TestBatch *self, bool_t condition, const char *label,
+              const char *file, int line);
 
     inert bool_t
     test_false(void *vself, bool_t condition, const char *pattern, ...);
@@ -64,10 +65,6 @@ class Lucy::Test::TestBatch inherits Lucy::Object::Obj {
     skip(void *vself, const char *pattern, ...);
 
     bool_t
-    VTest_True(TestBatch *self, bool_t condition, const char *pattern,
-               va_list args);
-
-    bool_t
     VTest_False(TestBatch *self, bool_t condition, const char *pattern,
                 va_list args);
 
@@ -94,8 +91,12 @@ class Lucy::Test::TestBatch inherits Lucy::Object::Obj {
 }
 
 __C__
+
+#define LUCY_TESTBATCH_TEST_TRUE(_batch, _condition, _label) \
+    lucy_TestBatch_test_true(_batch, _condition, _label, __FILE__, __LINE__)
+   
 #ifdef LUCY_USE_SHORT_NAMES
-  #define TEST_TRUE                    lucy_TestBatch_test_true
+  #define TEST_TRUE                    LUCY_TESTBATCH_TEST_TRUE
   #define TEST_FALSE                   lucy_TestBatch_test_false
   #define TEST_INT_EQ                  lucy_TestBatch_test_int_equals
   #define TEST_FLOAT_EQ                lucy_TestBatch_test_float_equals