drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Julian Hyde (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5290) Provide an option to build operator table once for built-in static functions and reuse it across queries.
Date Wed, 22 Feb 2017 19:51:44 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879088#comment-15879088
] 

Julian Hyde commented on DRILL-5290:
------------------------------------

This is a good idea. Just to clarify, the "static" functions don't need to be static fields
in the java class. They could be non-static if you choose, as long as you re-use the same
operator table instance. Calcite's operator table API gives you several ways to build and
combine operator tables, and reflection of (static) fields is not the only way. 

> Provide an option to build operator table once for built-in static functions and reuse
it across queries.
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-5290
>                 URL: https://issues.apache.org/jira/browse/DRILL-5290
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.9.0
>            Reporter: Padma Penumarthy
>            Assignee: Padma Penumarthy
>             Fix For: 1.10
>
>
> Currently, DrillOperatorTable which contains standard SQL operators and functions and
Drill User Defined Functions (UDFs) (built-in and dynamic) gets built for each query as part
of creating QueryContext. This is an expensive operation ( ~30 msec to build) and allocates
 ~2M on heap for each query. For high throughput, concurrent low latency operational queries,
we quickly run out of heap memory, causing JVM hangs. Build operator table once during startup
for static built-in functions and save in DrillbitContext, so we can reuse it across queries.
> Provide an system/session option to not use dynamic UDFs so we can use the operator table
saved in DrillbitContext and avoid building each time.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message