drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-5287) Provide option to skip updates of ephemeral state changes in Zookeeper
Date Mon, 27 Feb 2017 18:03:45 GMT

    [ https://issues.apache.org/jira/browse/DRILL-5287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15886231#comment-15886231

ASF GitHub Bot commented on DRILL-5287:

Github user ppadma commented on a diff in the pull request:

    --- Diff: exec/java-exec/src/main/java/org/apache/drill/exec/ExecConstants.java ---
    @@ -413,4 +413,8 @@
       String DYNAMIC_UDF_SUPPORT_ENABLED = "exec.udf.enable_dynamic_support";
true, true);
    +  String ZK_QUERY_STATE_UPDATE_KEY = "drill.exec.zk.query.state.update";
    --- End diff --
    I changed it to QUERY_TRANSIENT_STATE_UPDATE_KEY  and exec.query.progress.update. Please
review the new diffs. 

> Provide option to skip updates of ephemeral state changes in Zookeeper
> ----------------------------------------------------------------------
>                 Key: DRILL-5287
>                 URL: https://issues.apache.org/jira/browse/DRILL-5287
>             Project: Apache Drill
>          Issue Type: Bug
>    Affects Versions: 1.9.0
>            Reporter: Padma Penumarthy
>            Assignee: Padma Penumarthy
>             Fix For: 1.10
> We put transient profiles in zookeeper and update state as query progresses and changes
states. It is observed that this adds latency of ~45msec for each update in the query execution
path. This gets even worse when high number of concurrent queries are in progress. For concurrency=100,
the average query response time even for short queries  is 8 sec vs 0.2 sec with these updates
disabled. For short lived queries in a high-throughput scenario, it is of no value to update
state changes in zookeeper. We need an option to disable these updates for short running operational

This message was sent by Atlassian JIRA

View raw message