drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4132) Ability to submit simple type of physical plan directly to EndPoint DrillBit for execution
Date Wed, 27 Apr 2016 22:45:13 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15261109#comment-15261109

ASF GitHub Bot commented on DRILL-4132:

Github user hnfgns commented on a diff in the pull request:

    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/Foreman.java
    @@ -419,6 +425,64 @@ private void runPhysicalPlan(final PhysicalPlan plan) throws ExecutionSetupExcep
         logger.debug("Fragments running.");
    +  /**
    +   * This is a helper method to run query based on the list of PlanFragment that were
    +   * at some point of time
    +   * @param fragmentsList
    +   * @throws ExecutionSetupException
    +   */
    +  private void runFragment(List<PlanFragment> fragmentsList) throws ExecutionSetupException
    +    // need to set QueryId, MinorFragment for incoming Fragments
    +    PlanFragment rootFragment = null;
    +    boolean isFirst = true;
    +    final List<PlanFragment> planFragments = Lists.newArrayList();
    +    final int fragmentsCount = fragmentsList.size();
    +    for (PlanFragment myFragment : fragmentsList) {
    +      final FragmentHandle handle = myFragment.getHandle();
    +      // for split plan number of minor fragments will be always one,
    +      // but minor fragmentId may not be 0, as it is one of the minor fragments from
split plan
    +      final int minorFragment = (fragmentsCount == 1) ? 0 : handle.getMinorFragmentId();
    --- End diff --
    Even though I understand that a split plan consists of a single minor fragment, I wonder
if there is a better way to handle minor fragment numbering. Why do not we just re-number
minor fragments during planning/splitting phase at the first place instead of relying on fragmentsCount
== 1 check?

> Ability to submit simple type of physical plan directly to EndPoint DrillBit for execution
> ------------------------------------------------------------------------------------------
>                 Key: DRILL-4132
>                 URL: https://issues.apache.org/jira/browse/DRILL-4132
>             Project: Apache Drill
>          Issue Type: New Feature
>          Components: Execution - Flow, Execution - RPC, Query Planning & Optimization
>            Reporter: Yuliya Feldman
>            Assignee: Yuliya Feldman
> Today Drill Query execution is optimistic and stateful (at least due to data exchanges)
- if any of the stages of query execution fails whole query fails. If query is just simple
scan, filter push down and project where no data exchange happens between DrillBits there
is no need to fail whole query when one DrillBit fails, as minor fragments running on that
DrillBit can be rerun on the other DrillBit. There are probably multiple ways to achieve this.
This JIRA is to open discussion on: 
> 1. agreement that we need to support above use case 
> 2. means of achieving it.

This message was sent by Atlassian JIRA

View raw message