drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Parth Chandra (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-5223) Drill should ensure balanced workload assignment at node level in order to get better query performance
Date Wed, 08 Feb 2017 05:17:41 GMT

     [ https://issues.apache.org/jira/browse/DRILL-5223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Parth Chandra updated DRILL-5223:
---------------------------------
    Labels: ready-to-commit  (was: )

> Drill should ensure balanced workload assignment at node level in order to get better
query performance
> -------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-5223
>                 URL: https://issues.apache.org/jira/browse/DRILL-5223
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>            Reporter: Jinfeng Ni
>            Assignee: Padma Penumarthy
>              Labels: ready-to-commit
>
> Drill's work assignment logic currently aims to achieve workload balance across different
minor fragment (or slices) and honor data affinity in order to get as much local read as possible.

> However, when the # of work units could not be evenly divided by # of minor fragments,
the remaining work units would tender to go to the first subset of drill endpoints. This means
the drill endpoints assigned with the remaining work units could have larger workload than
the rest of them.  When MuxExchange is enabled (by default), all the minor fragments on the
same node have to send data to a single Muxer per node, and unbalanced workload assignment
at node level could impact query elapse time.  which is essentially decided by the slowest
drill endpoint.
> Some prototype experimental run shows that with more balanced workload assignment, Drill
shows quite significant improvement for most of TPC-H queries. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message