drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacques Nadeau (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-60) multiple query support
Date Sun, 04 Jan 2015 21:27:40 GMT

     [ https://issues.apache.org/jira/browse/DRILL-60?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jacques Nadeau updated DRILL-60:
    Component/s: Execution - Operators

> multiple query support
> ----------------------
>                 Key: DRILL-60
>                 URL: https://issues.apache.org/jira/browse/DRILL-60
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Execution - Operators
>            Reporter: Lisen Mu
>             Fix For: Future
>         Attachments: DRILL-60.0.patch.txt
> parse/optimize several SQL(or other QL)s into one logical plan and execute them once.
> * use case
> If a report generating system is used as a frontend system of drill instead of a command
line, There would be a set of similar SQL to execute to generate a report(consider each day's
DAU statistics with geoip distribution(groupby) for 7 continuous days). If processed individually,
plans for SQLs in this kind of set are similar(scan pv log in these days) or same(scan GeoIP
table) in majority. If processed together, duplicated scan(GeoIP) could be eliminated to reduce
I/O, or similar scan(pv) could be merged to achieve better throughput, and provides chances
of other intermediate result reuse.
> * possibility
> Query in a QL is represented as a tree. Drill models logical plan and physical plan as
DAG, which is more expressive than tree. More precisely, a set of trees(with common nodes
or common edges) can be union-ed into a dag. So, the current drill logical/physical plan model
supports multiple query already. Each sql would have a corresponding StoreOP as its result.
> * problems
> To do so, physical operators must support multiple iteration over its output. That would
have side effect on mem management.

This message was sent by Atlassian JIRA

View raw message