drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5325) Implement sub-operator unit tests for managed external sort
Date Tue, 06 Jun 2017 22:41:18 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16039803#comment-16039803
] 

ASF GitHub Bot commented on DRILL-5325:
---------------------------------------

Github user paul-rogers commented on a diff in the pull request:

    https://github.com/apache/drill/pull/808#discussion_r119226233
  
    --- Diff: common/src/main/java/org/apache/drill/test/SecondaryTest.java ---
    @@ -0,0 +1,69 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one
    + * or more contributor license agreements.  See the NOTICE file
    + * distributed with this work for additional information
    + * regarding copyright ownership.  The ASF licenses this file
    + * to you under the Apache License, Version 2.0 (the
    + * "License"); you may not use this file except in compliance
    + * with the License.  You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +package org.apache.drill.test;
    +
    +/**
    + * Label for Drill secondary tests. A secondary test is one that is omitted from
    + * the normal Drill build because:
    + * <ul>
    + * <li>It is slow</li>
    + * <li>It tests particular functionality which need not be tested on every
    + * build.</li>
    + * <li>It is old, but still worth running some times.</li>
    + * <li>It requires specialized setup and/or runs on specific platforms.</li>
    + * </ul>
    + *
    + * To mark a test as secondary, do either:<pre>
    + * {@literal @}Category(SecondaryTest.class)
    + * class MyTest {
    + *    ...
    + * }
    + * </pre></code>Or:<pre><code>
    + * class MyTest {
    + *   {@literal @}Category(SecondaryTest.class)
    + *   public void slowTest() { ... }
    + * }
    + * </code></pre>Maven is configured as follows:<pre><code>
    + *    &lt;plugin>
    + *      &lt;artifactId>maven-surefire-plugin&lt;/artifactId>
    + *      ...
    + *      &lt;configuration>
    + *        ...
    + *        &lt;excludedGroups>org.apache.drill.test.SecondaryTest&lt;/excludedGroups>
    + *      &lt;/configuration>
    + *      ...
    + *    &lt;/plugin></code></pre>
    + *  To run the secondary tests (preliminary):<pre><code>
    + *  > mvn surefire:test -Dgroups=org.apache.drill.test.SecondaryTest -DexcludedGroups=</code></pre>
    + *  The above says to run the secondary test and exclude nothing. The exclusion
    + *  is required to override the default exclusions: skip that parameter and Maven will
    + *  blindly try to run all tests annotated with the SecondaryTest category except
    + *  for those annotated with the SecondaryTest category, which is not very helpful...
    + *  <p>
    + *  Note that <tt>java-exec</tt> (only) provides a named execution to run
large tests:
    + *  <p>
    + *  <tt>mvn surefire:test@include-large-tests</tt>
    + *  <p>
    + *  However, the above does not work. Nor did it work to include the category in
    + *  a profile as described earlier. At present, there is no known way to run just
    --- End diff --
    
    Sorry, I'll make the comment clearer. I'm trying to say that we claim we have a way to
run secondary (slower) tests separate from the bulk of our unit tests. But, when I tried to
use it, it turns out that the solution does not work. As a result, I was forced to include
certain (slow) tests in the bulk of our unit tests.


> Implement sub-operator unit tests for managed external sort
> -----------------------------------------------------------
>
>                 Key: DRILL-5325
>                 URL: https://issues.apache.org/jira/browse/DRILL-5325
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Tools, Build & Test
>    Affects Versions: 1.11.0
>            Reporter: Paul Rogers
>            Assignee: Paul Rogers
>             Fix For: 1.11.0
>
>
> Validate the proposed sub-operator test framework, by creating low-level unit tests for
the managed version of the external sort.
> The external sort has a small number of existing tests, but those tests are quite superficial;
the "managed sort" project found many bugs. The managed sort itself was tested with ad-hoc
system-level tests created using the new "cluster fixture" framework. But, again, such tests
could not reach deep inside the sort code to exercise very specific conditions.
> As a result, we spent far too much time using QA functional tests to identify specific
code issues.
> Using the sub-opeator unit test framework, we can instead test each bit of functionality
at the unit test level.
> If doing so works, and is practical, it can serve as a model for other operator testing
projects.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message