couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (COUCHDB-1963) Move from etap to eunit
Date Mon, 23 Jun 2014 12:22:25 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-1963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14040674#comment-14040674
] 

ASF GitHub Bot commented on COUCHDB-1963:
-----------------------------------------

GitHub user kxepal opened a pull request:

    https://github.com/apache/couchdb/pull/253

    Replace etap tests with eunit

    This PR replaces etap test with eunit.
    
    Issues with etap:
    
    1. It's etap (:
    2. Almost for all suites each next test depends from the state of previous one
    3. Lack of integration with rebar out of box
    4. Badly organization
    
    Goals:
    
    1. Refresh test suite: split and isolate each test case
    2. Find and get rid heisenbugs
    3. Make test suite ready for CouchDB 2.0 build system
        
    Considerations:
    
    1. Each target test function name reflects the test goal
    2. Each target test function starts with `should_` prefix to 1) separate test functions
from helper ones 2) make names more clean
    3. Otherwise test has description or reference to couchdb issue
    4. Test suites which tests specific module named as this module is
    5. Otherwise they starts with more common "couchdb_" prefix
    6. Tests output should be human friendly
    
    Fixed issues (at least):
    
    - Timeouts are reduced to meanful values, all timer:sleep are removed unless they are
required for test logic
    - 073-changes.t: heartbeats count is very depended from host's performance
    - 090-task-status.t: get_task_prop silently resets Acc if searched task isn't last in
the list
    - 140-attachments-comp.t: zlib:gzip only worked for small attachments
    - 201-view-group-shutdown.t: race condition for {error, all_dbs_active} assertion check.
Previously was solved by timer:sleep.
    
    Some stats:
    
    - CouchDB: 
      - Etap:
        - Files=52, Tests=1219
        - time make check  107.92s user 11.17s system 35% cpu 5:38.20 total
      - Eunit:
        - Files=29, Tests=693
        - time make check  74.41s user 11.68s system 51% cpu 2:48.48 total
    - couch_mrview:
      - Etap:
        - Files=7, Tests=41
        - time make check  3.02s user 0.79s system 73% cpu 5.191 total
      - Eunit:
        - Files=7, Tests=39
        - time make check  1.48s user 0.32s system 52% cpu 3.405 total
    - couch_replicator:
      - Etap:
        - Files=7, Tests=1832
        - time make check  115.11s user 13.94s system 81% cpu 2:37.98 total
      - Eunit:
        - Files=7, Tests=127
        - time make check  133.73s user 15.99s system 89% cpu 2:47.51 total
    
    (Notice: as for etap each "test" actually means the assertion)
    
    Expected output: https://www.friendpaste.com/2MCgzw3g18ATOWyLGBSQwZ
    
    Notice: I'd ran tests both on fast and slow environment (single cored virtualbox with
40% cpu cap), so execution timeouts fits both. However, I'm not sure about all possible configurations,
so if any test fails with *timeout* for you please include in your report your system configuration
- it
        
    Notice: [error] messages in output is *FINE* - they happens because `log=none` doesn't
disables logging completely, allowing error messages to be emit out.
    
    P.S. All commits after "Goodbye etap!" will be squashed before merge.
    
    COUCHDB-1963

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/apache/couchdb 1963-eunit

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/couchdb/pull/253.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #253
    
----
commit c9c457607ad61b6c61bb51df8ebda36bcc809e3f
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-15T20:08:36Z

    Add run script to execute eunit tests
    
    Usage is the same as for test/etap/run:
    
        ./test/couchdb/run -v ${PATH}
    
    -v runs in verbose mode, as etap does. Also, you can use make for that:
    
        make check-eunit
    
    which will run tests everywhere in project where Makefile contains
    check-eunit subcommand definition.
    
    The ${PATH} thing could be single file or directory. The latter should
    contains *_tests.erl files which would be compiled and executed by
    eunit.
    
    The *_tests.erl
    
    The reason of compiling on run instead of using autoconf for that is
    to simplify tests developing and avoid situations, when you'd fixed
    the test or add new one, but forgot to remove/compile beam file.
    
    All test_*.beam files are been stored in test/couchdb/ebin directory,
    the temporary test files will be places to test/couchdb/temp one.
    Both directories will be removed by make clean/distclean command.

commit 47e61617ff0f444a0e74f9fce58562e3f4e6c3d5
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-15T23:46:16Z

    Add common header for eunit test files

commit 33a95efbeb0613b121f4c36ef36092f0893ee00d
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-16T00:45:38Z

    Port 001-load.t etap test suite to eunit

commit 01d6bca33c865bfce5798da1c9b492a1fa6bfe6d
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-16T01:19:05Z

    Port 002-icu-driver.t etap test suite to eunit
    
    See setup/0 comment for specific info about loading
    couch_icu_driver with eunit.

commit 5bbb1140195f7a8e39bd36d4b422090fe3af4442
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-16T14:38:56Z

    Port 010-file-basics.t and 011-file-headers.t etap test suites to eunit
    
    Both merged into single suite since they tests single target and shares
    common bits.

commit c3dc98de0ad62dfd6021777b27632e48ee30de6d
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-16T23:28:14Z

    Port 020-btree-basics.t and 021-btree-reductions.t etap suites to eunit
    
    Both merged into single suite since they tests single target and shares
    common bits.

commit ed7b576a7267227d73a412c0066a04efefb5c955
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-16T23:42:01Z

    Port 030-doc-from-json.t and 031-doc-to-json.t etap suites to eunit
    
    Both merged into single suite since they tests single target and shares
    common bits.

commit 33f5575e192f03cb450c332f6eb202e33f1b9565
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-17T00:16:38Z

    Port 040-util.t etap test suite to eunit

commit 332596355db5a4e2d24574a14ec89c6fc0f34ed7
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-17T00:23:25Z

    Port 041-uuid.t etap test suite to eunit
    
    Config files are removed in favor of using couch_config API instead.

commit 6162c9541f46a23cd079f7901a4b0b260fc1b397
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-18T10:04:08Z

    Port 042-work-queue.t etap test suite to eunit
    
    Etap tests were made in flow style, testing the same things multiple
    times without real need. For eunit they are split into small test cases
    to focus on testing goals.
    
    Timeout on receive is decreased from 3000 to 100.

commit de228dc0629aadd210e23cad055f9a68fbea5055
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-18T10:35:58Z

    Port 043-find-in-binary.t etap test suite to eunit
    
    It been merged into couch_util_tests suite.

commit 31c6634c32da99de14b3c00e9787a70ea4c0b78d
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-18T10:45:05Z

    Port 050-stream.t etap test suite to eunit

commit d00e3a278769692d95db340c2e7fa5c3d1f07ff8
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-18T10:53:03Z

    Port 06[0-5]-kt-*.t etap test suites to eunit
    
    All merged into single suite since they're test the same module.

commit 3504d9882fbc66f06cd0309e137856ca9e26bfff
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-19T01:36:06Z

    Port 070-couch-db.t etap test suite to eunit
    
    Fix ERL_LIB environment variable setting. Add ?tempdb macros for
    unique temporary database name generation.

commit d22c1e1c1ac6f066a9d03fadfbd6776a810e71fc
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-20T03:16:41Z

    Port 072-cleanup.t etap test suite to eunit
    
    requests functions from test_util were moved to test_request module
    with nicer and simpler API.

commit 2c6c7249849c6f24cf822839647dc433c77a70eb
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-20T03:19:22Z

    Port 073-changes.t etap test suite to eunit
    
    For heartbeats test they don't being counted anymore since their
    amount is heavy depends from the overall system performance and some
    assertions may fail or not because of that. Instead of this, we just
    ensure that their amount is going to increase over the time.

commit aff968819775a78a67b090dfa97f6e5df3f2e80b
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-21T14:58:12Z

    Port 074-doc-update-conflicts.t etap test suite to eunit
    
    Timeout decreased, added 10K clients case

commit 085e0ed289eafbae9a5e1c526cc25a2b39d9008a
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-22T09:07:24Z

    Port 075-auth-cache.t etap test suite to eunit
    
    Timeouts are removed.

commit e656e8d710e565fc0c7aa0420dee2b407dadd3f8
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-22T16:17:47Z

    Port 076-file-compression.t etap test suite to eunit
    
    The original test suite was decoupled into compaction and comparison
    cases.

commit d01b88abbecbe7de2ddc51d0fc4dd88f3b32816d
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-22T16:31:09Z

    Port 077-couch-db-fast-db-delete-create.t etap test suite to eunit
    
    Merged into couch_db_tests suite.

commit c44c8d13c1ab1a88c08e3f3059138fae09430314
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-23T05:29:22Z

    Port 080-config-get-set.t etap test suite to eunit

commit 09a0e6d369f45e97874698166cb0efbdaced48a5
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-25T18:02:19Z

    Port 081-config-override.t etap test suite to eunit
    
    Merged into couch_config_tests suite.
    Setup fixtures.

commit 806f13bbd8d8985eae044b4650ca630ecb5fd13e
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-26T05:26:22Z

    Port 082-config-register.t etap test suite to eunit
    
    Merged into couch_config_tests suite.

commit ddbbd519b193bc0fb1d7ca124b233790979f4437
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-26T05:46:06Z

    Port 083-config-no-files.t etap test suite to eunit
    
    Merged into couch_config_tests suite.

commit 9227502e1f00920de4db3e177e76551563d99826
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-26T16:23:41Z

    Port 090-task-status.t etap test suite to eunit
    
    Split huge test case into multiple ones. Fix issue with get_task_prop
    when Acc may be reset if searched task isn't last in the list.

commit fad32ab98af0abde6ea01694136ee1f79844f914
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-26T16:59:45Z

    Port 100-ref-counter.t etap test suite to eunit

commit 8bc52a1171f2eb87a8523c8746399b1fd739a33c
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-26T19:57:38Z

    Port 120-stats-collect.t etap test suite to eunit

commit 19c5402311dde98a79f870d6fc3814de5503498d
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-27T14:27:53Z

    Port 121-stats-aggregates.t etap test suite to eunit
    
    Merged into couch_stats_tests suite.

commit d0e50a0843cec3da88908b04585c750c878451a5
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-05-28T01:32:11Z

    Port 130-attachments-md5.t etap test suite to eunit
    
    Add random document id generator macros.
    Have to use handmade http client instead of ibrowse since it makes too
    complicated sending chunked requests.

commit 5726d0689d4b9e37df8476726e97ee6fa92c46bb
Author: Alexander Shorin <kxepal@apache.org>
Date:   2014-06-02T21:51:46Z

    Port 140-attachments-comp.t etap test suite to eunit
    
    - Merge into couchdb_attachments_tests suite;
    - Add PUT requests to test_request util;
    - Remove dependency from files outside fixtures directory;
    - Group test cases to reduce amount of duplicate code;
    - Fix hidden issue with gzip encoding: for encoding_length stub info
      check using zlib:gzip on 2KiB+ files leads to mismatch by 2-4 bytes
      and this difference grows with file size. Using gzip fun code from
      couch_stream solves the issue.

----


> Move from etap to eunit
> -----------------------
>
>                 Key: COUCHDB-1963
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1963
>             Project: CouchDB
>          Issue Type: Improvement
>          Components: Test Suite
>            Reporter: Joan Touzet
>            Assignee: Alexander Shorin
>              Labels: etap, eunit, test
>
> Pursuant to an IRC/Twitter discussion, it seems that there is some
> support ([~pjdavis1970], [~janl]) to move off of etap. Motivations include etap being
unmaintained for 2+ years, eunit shipping with Erlang, and least importantly etap tests being
completely broken on the Windows build.
> There are only 63 .t files in the entire source tree; hopefully this is work that can
be weekend-tackled by one person, or could be tag-teamed with little friction.
> There was a subsequent discussion about the current, existing JS tests; while there are
issues with these and room for improvement, this bug is *strictly* about our white-box etap
unit tests and larger multi-instance tests that may not best be accomplished by a JS framework.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message