drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5126) Provide simplified, unified "cluster fixture" for tests
Date Sat, 07 Jan 2017 01:16:00 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15806386#comment-15806386
] 

ASF GitHub Bot commented on DRILL-5126:
---------------------------------------

Github user sohami commented on a diff in the pull request:

    https://github.com/apache/drill/pull/710#discussion_r95019120
  
    --- Diff: exec/java-exec/src/test/java/org/apache/drill/test/FixtureBuilder.java ---
    @@ -0,0 +1,251 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one
    + * or more contributor license agreements.  See the NOTICE file
    + * distributed with this work for additional information
    + * regarding copyright ownership.  The ASF licenses this file
    + * to you under the Apache License, Version 2.0 (the
    + * "License"); you may not use this file except in compliance
    + * with the License.  You may obtain a copy of the License at
    + *
    + * http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + ******************************************************************************/
    +package org.apache.drill.test;
    +
    +import java.util.ArrayList;
    +import java.util.List;
    +import java.util.Properties;
    +
    +import org.apache.drill.exec.ExecConstants;
    +import org.apache.drill.exec.ZookeeperHelper;
    +
    +/**
    + * Build a Drillbit and client with the options provided. The simplest
    + * builder starts an embedded Drillbit, with the "dfs_test" name space,
    + * a max width (parallelization) of 2.
    + */
    +
    +public class FixtureBuilder {
    +
    +  public static class RuntimeOption {
    +    public String key;
    +    public Object value;
    +
    +    public RuntimeOption(String key, Object value) {
    +      this.key = key;
    +      this.value = value;
    +    }
    +  }
    +
    +  // Values in the drill-module.conf file for values that are customized
    +  // in the defaults.
    +
    +  public static final int DEFAULT_ZK_REFRESH = 500; // ms
    +  public static final int DEFAULT_SERVER_RPC_THREADS = 10;
    +  public static final int DEFAULT_SCAN_THREADS = 8;
    +
    +  public static Properties defaultProps() {
    +    Properties props = new Properties();
    +    props.putAll(ClusterFixture.TEST_CONFIGURATIONS);
    +    return props;
    +  }
    +
    +  String configResource;
    +  Properties configProps;
    +  boolean enableFullCache;
    +  List<RuntimeOption> sessionOptions;
    +  List<RuntimeOption> systemOptions;
    +  int bitCount = 1;
    +  String bitNames[];
    +  int zkCount;
    +  ZookeeperHelper zkHelper;
    +
    +  /**
    +   * Use the given configuration properties to start the embedded Drillbit.
    +   * @param configProps a collection of config properties
    +   * @return this builder
    +   * @see {@link #configProperty(String, Object)}
    +   */
    +
    +  public FixtureBuilder configProps(Properties configProps) {
    +    this.configProps = configProps;
    +    return this;
    +  }
    +
    +  /**
    +   * Use the given configuration file, stored as a resource, to start the
    +   * embedded Drillbit. Note that the resource file should have the two
    +   * following settings to work as a test:
    +   * <pre><code>
    +   * drill.exec.sys.store.provider.local.write : false,
    +   * drill.exec.http.enabled : false
    +   * </code></pre>
    +   * It may be more convenient to add your settings to the default
    +   * config settings with {@link #configProperty(String, Object)}.
    +   * @param configResource path to the file that contains the
    +   * config file to be read
    +   * @return this builder
    +   * @see {@link #configProperty(String, Object)}
    +   */
    +
    +  public FixtureBuilder configResource(String configResource) {
    +
    +    // TypeSafe gets unhappy about a leading slash, but other functions
    +    // require it. Silently discard the leading slash if given to
    +    // preserve the test writer's sanity.
    +
    +    this.configResource = ClusterFixture.trimSlash(configResource);
    +    return this;
    +  }
    +
    +  /**
    +   * Add an additional boot-time property for the embedded Drillbit.
    +   * @param key config property name
    +   * @param value property value
    +   * @return this builder
    +   */
    +
    +  public FixtureBuilder configProperty(String key, Object value) {
    +    if (configProps == null) {
    +      configProps = defaultProps();
    +    }
    +    configProps.put(key, value.toString());
    +    return this;
    +  }
    +
    +   /**
    +   * Provide a session option to be set once the Drillbit
    +   * is started.
    +   *
    +   * @param key the name of the session option
    +   * @param value the value of the session option
    +   * @return this builder
    +   * @see {@link ClusterFixture#alterSession(String, Object)}
    +   */
    +
    +  public FixtureBuilder sessionOption(String key, Object value) {
    +    if (sessionOptions == null) {
    +      sessionOptions = new ArrayList<>();
    +    }
    +    sessionOptions.add(new RuntimeOption(key, value));
    +    return this;
    +  }
    +
    +  /**
    +   * Provide a system option to be set once the Drillbit
    +   * is started.
    +   *
    +   * @param key the name of the system option
    +   * @param value the value of the system option
    +   * @return this builder
    +   * @see {@link ClusterFixture#alterSystem(String, Object)}
    +   */
    +
    +  public FixtureBuilder systemOption(String key, Object value) {
    +    if (systemOptions == null) {
    +      systemOptions = new ArrayList<>();
    +    }
    +    systemOptions.add(new RuntimeOption(key, value));
    +    return this;
    +  }
    +
    +  /**
    +   * Set the maximum parallelization (max width per node). Defaults
    +   * to 2.
    +   *
    +   * @param n the "max width per node" parallelization option.
    +   * @return this builder
    +   */
    +  public FixtureBuilder maxParallelization(int n) {
    +    return sessionOption(ExecConstants.MAX_WIDTH_PER_NODE_KEY, n);
    --- End diff --
    
    you should return Builder here and set the sessionOption ?


> Provide simplified, unified "cluster fixture" for tests
> -------------------------------------------------------
>
>                 Key: DRILL-5126
>                 URL: https://issues.apache.org/jira/browse/DRILL-5126
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Tools, Build & Test
>    Affects Versions: 1.9.0
>            Reporter: Paul Rogers
>            Assignee: Paul Rogers
>            Priority: Minor
>
> Drill provides a robust selection of test frameworks that have evolved to satisfy the
needs of a variety of test cases. For newbies, however, the result is a bewildering array
of ways to do basically the same thing: set up an embedded Drill cluster, run queries and
check results.
> Further, some key test settings are distributed: some are in the pom.xml file, some in
config files stored as resources, some in hard-coded settings in base test classes.
> Also, some test base classes helpfully set up a test cluster, but then individual tests
need a different config, so they immediately tear down the default cluster and create a new
one.
> This ticket proposes a new test framework, available for new tests, that combines the
best of the existing test frameworks into a single, easy-to-use package.
> * Builder for the cluster
> * Accept config-time options
> * Accept run-time session and system options
> * Specify number of Drillbits
> * Simplified API for the most common options
> * AutoCloseable for use in try-with-resources statements
> * Integration with existing test builder classes
> And so on.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message