Mailing-List: contact dev-help@jackrabbit.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@jackrabbit.apache.org
Message-ID: <1428409514.1204716463283.JavaMail.jira@brutus>
Date: Wed, 5 Mar 2008 03:27:43 -0800 (PST)
From: "Jukka Zitting (JIRA)" <jira@apache.org>
To: dev@jackrabbit.apache.org
Subject: [jira] Commented: (JCR-1452) Make use of jackrabbit.test.scale in
 test cases
In-Reply-To: <1388321621.1204635040900.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/JCR-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12575308#action_12575308 ] 

Jukka Zitting commented on JCR-1452:
------------------------------------

> It would be strange to write three distinct tests, one with 500 MB, one with 50 KB, and another for 50 bytes.

I'm all OK for those tests to share code for example in a common base class, i.e. have an AbstractBinaryTest base class and subclasses like LargerThanMemoryBinaryTest, NormalBinaryTest, InlineBinaryTest for the distinct cases. Each test has a different purpose and a different target code path, and so I very much disagree that they are just differently scaled versions of the same test.

> > we shouldn't just test things because we can.
> 
> There are two kinds of tests: tests that are written after finding a bug, and tests that are written to prevent bugs.

Agreed. My point is that we should rather use a test coverage tool or something similar to identify what tests we should add instead of just blindly scaling existing tests up or down.

> Also, the TCK is not (yet) run regularly AFAIK.

Not the official TCK, but all the tests in it (jackrabbit-jcr-tests) are run per each "mvn install" and CI build.

> I prefer to use the 'scale' system property

OK, but how should I use that? What does scale=1 mean? If I want to scale up, should I set it to scale=2, scale=10, or scale=1000? What do those settings mean, and how can I be sure that the tests I'm running are reasonably scaled? Is the scale setting consistent across different tests?

For example, to cover the 500MB, 50kB, and 50b cases for the binary test mentioned above, should I use scale=50, scale=50000, and scale=500000000? Without knowing the details of the data store implementation and the associated thresholds for example for inlining the bytes, how can I choose reasonable scales so that all relevant scenarios get tested. I can't just blindly start testing with scale 1, 2, 3, ...

The writer of the test case has always the best knowledge of what are the important test cases and it should be up to him/her to specify the appropriate settings to cover those cases.


> Make use of jackrabbit.test.scale in test cases
> -----------------------------------------------
>
>                 Key: JCR-1452
>                 URL: https://issues.apache.org/jira/browse/JCR-1452
>             Project: Jackrabbit
>          Issue Type: Improvement
>          Components: jackrabbit-core
>            Reporter: Marcel Reutegger
>            Priority: Minor
>             Fix For: 1.5
>
>
> There are already a number of longer running test cases in jackrabbit-core, but they are all disabled because they otherwise make building jackrabbit-core a very long task.
> Those tests should make use of the jackrabbit.test.scale property and per default (scale = 1) run within a short time.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.