jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jukka Zitting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (JCR-1452) Make use of jackrabbit.test.scale in test cases
Date Wed, 05 Mar 2008 11:27:43 GMT

    [ https://issues.apache.org/jira/browse/JCR-1452?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12575308#action_12575308
] 

Jukka Zitting commented on JCR-1452:
------------------------------------

> It would be strange to write three distinct tests, one with 500 MB, one with 50 KB, and
another for 50 bytes.

I'm all OK for those tests to share code for example in a common base class, i.e. have an
AbstractBinaryTest base class and subclasses like LargerThanMemoryBinaryTest, NormalBinaryTest,
InlineBinaryTest for the distinct cases. Each test has a different purpose and a different
target code path, and so I very much disagree that they are just differently scaled versions
of the same test.

> > we shouldn't just test things because we can.
> 
> There are two kinds of tests: tests that are written after finding a bug, and tests that
are written to prevent bugs.

Agreed. My point is that we should rather use a test coverage tool or something similar to
identify what tests we should add instead of just blindly scaling existing tests up or down.

> Also, the TCK is not (yet) run regularly AFAIK.

Not the official TCK, but all the tests in it (jackrabbit-jcr-tests) are run per each "mvn
install" and CI build.

> I prefer to use the 'scale' system property

OK, but how should I use that? What does scale=1 mean? If I want to scale up, should I set
it to scale=2, scale=10, or scale=1000? What do those settings mean, and how can I be sure
that the tests I'm running are reasonably scaled? Is the scale setting consistent across different
tests?

For example, to cover the 500MB, 50kB, and 50b cases for the binary test mentioned above,
should I use scale=50, scale=50000, and scale=500000000? Without knowing the details of the
data store implementation and the associated thresholds for example for inlining the bytes,
how can I choose reasonable scales so that all relevant scenarios get tested. I can't just
blindly start testing with scale 1, 2, 3, ...

The writer of the test case has always the best knowledge of what are the important test cases
and it should be up to him/her to specify the appropriate settings to cover those cases.


> Make use of jackrabbit.test.scale in test cases
> -----------------------------------------------
>
>                 Key: JCR-1452
>                 URL: https://issues.apache.org/jira/browse/JCR-1452
>             Project: Jackrabbit
>          Issue Type: Improvement
>          Components: jackrabbit-core
>            Reporter: Marcel Reutegger
>            Priority: Minor
>             Fix For: 1.5
>
>
> There are already a number of longer running test cases in jackrabbit-core, but they
are all disabled because they otherwise make building jackrabbit-core a very long task.
> Those tests should make use of the jackrabbit.test.scale property and per default (scale
= 1) run within a short time.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message