drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5459) Extend physical operator test framework to test mini plans consisting of multiple operators
Date Fri, 12 May 2017 23:06:04 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16008881#comment-16008881
] 

ASF GitHub Bot commented on DRILL-5459:
---------------------------------------

Github user jinfengni commented on a diff in the pull request:

    https://github.com/apache/drill/pull/823#discussion_r116337155
  
    --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/physical/unit/TestMiniPlan.java
---
    @@ -0,0 +1,200 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one
    + * or more contributor license agreements.  See the NOTICE file
    + * distributed with this work for additional information
    + * regarding copyright ownership.  The ASF licenses this file
    + * to you under the Apache License, Version 2.0 (the
    + * "License"); you may not use this file except in compliance
    + * with the License.  You may obtain a copy of the License at
    + * <p/>
    + * http://www.apache.org/licenses/LICENSE-2.0
    + * <p/>
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.drill.exec.physical.unit;
    +
    +import com.google.common.collect.Lists;
    +import org.apache.drill.common.types.TypeProtos;
    +import org.apache.drill.common.util.FileUtils;
    +import org.apache.drill.exec.physical.config.Filter;
    +import org.apache.drill.exec.physical.config.UnionAll;
    +import org.apache.drill.exec.record.BatchSchema;
    +import org.apache.drill.exec.record.RecordBatch;
    +import org.apache.drill.exec.store.dfs.DrillFileSystem;
    +import org.apache.drill.test.rowSet.SchemaBuilder;
    +import org.apache.hadoop.conf.Configuration;
    +import org.apache.hadoop.fs.FileSystem;
    +import org.junit.BeforeClass;
    +import org.junit.Ignore;
    +import org.junit.Test;
    +
    +import java.util.Collections;
    +import java.util.List;
    +
    +public class TestMiniPlan extends MiniPlanUnitTestBase {
    +
    +  protected static DrillFileSystem fs;
    +
    +  @BeforeClass
    +  public static void initFS() throws Exception {
    +    Configuration conf = new Configuration();
    +    conf.set(FileSystem.FS_DEFAULT_NAME_KEY, "local");
    --- End diff --
    
    Modified per review comment. Also modified several existing testcases which used "local".



> Extend physical operator test framework to test mini plans consisting of multiple operators
> -------------------------------------------------------------------------------------------
>
>                 Key: DRILL-5459
>                 URL: https://issues.apache.org/jira/browse/DRILL-5459
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Tools, Build & Test
>            Reporter: Jinfeng Ni
>            Assignee: Jinfeng Ni
>              Labels: ready-to-commit
>
> DRILL-4437 introduced a unit test framework to test a non-scan physical operator. A JSON
reader is implicitly used to specify the inputs to the physical operator under test. 
> There are needs to extend such unit test framework for two scenarios.
> 1. We need a way to test scan operator with different record readers. Drill supports
a variety of data source, and it's important to make sure every record reader work properly
according to the protocol defined.
> 2. We need a way to test a so-called mini-plan (aka plan fragment) consisting of multiple
non-scan operators. 
> For the 2nd need, an alternative is to leverage SQL statement and query planner. However,
such approach has a direct dependency on query planner; 1) any planner change may impact the
testcase and lead to a different plan, 2) it's not always easy job to force the planner to
get a desired plan fragment for testing.
> In particular, it would be good to have a relatively easy way to specify a mini-plan
with a couple of targeted physical operators. 
> This JIRA is created to track the work to extend the unit test framework in DRILL-4437.
>  
> Related work: DRILL-5318 introduced a sub-operator test fixture, which mainly targeted
to test at sub-operator level. The framework in DRILL-4437 and the extension would focus on
operator level, or multiple operator levels, where execution would go through RecordBatch's
API call. 
> Same as DRILL-4437, we are going to use mockit to mock required objects such fragment
context, operator context etc. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message