hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <>
Subject [jira] [Commented] (HIVE-17508) Implement pool rules and triggers based on counters
Date Wed, 27 Sep 2017 18:13:01 GMT


Prasanth Jayachandran commented on HIVE-17508:

Like the idea of f(<Query>, <Rule>) -> <Action>, all applied in WM. The
initial issue was with the poll semantics for getting the progress in driver code which will
essentially loop in TezJobMonitor. I will probably add some listener for counters, TezJobMonitor
will then publish to those listeners (WM). I will address your comments in the next patch.

As far as the grammar for expression goes, I don't think a full fledged grammar would be required.
Isn't there a jira already to handle the frontend? I added this primarily by seeing the schema
in the metastore which stores the expression as varchar. If frontend grammar already exists
then I can reuse, else I will add the grammar in a follow up and integrate it with this patch.
The only expression of interest is greater than '>'. Also having >1 rule is equivalent
of  disjunctive evaluation. Any rule violation will trigger action instead of all rule violation.
IMHO conjunctive evaluation can be added later with full fledged grammar.

> Implement pool rules and triggers based on counters
> ---------------------------------------------------
>                 Key: HIVE-17508
>                 URL:
>             Project: Hive
>          Issue Type: Sub-task
>    Affects Versions: 3.0.0
>            Reporter: Prasanth Jayachandran
>            Assignee: Prasanth Jayachandran
>         Attachments: HIVE-17508.1.patch, HIVE-17508.2.patch, HIVE-17508.3.patch, HIVE-17508.3.patch,
HIVE-17508.WIP.2.patch, HIVE-17508.WIP.patch
> Workload management can defined Rules that are bound to a resource plan. Each rule can
have a trigger expression and an action associated with it. Trigger expressions are evaluated
at runtime after configurable check interval, based on which actions like killing a query,
moving a query to different pool etc. will get invoked. Simple rule could be something like
> {code}
> CREATE RULE slow_query IN resource_plan_name
> WHEN execution_time_ms > 10000
> MOVE TO slow_queue
> {code}

This message was sent by Atlassian JIRA

View raw message