hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-4447) Provide a mechanism to represent complex filters and parse them at the REST layer
Date Fri, 29 Apr 2016 21:33:13 GMT

     [ https://issues.apache.org/jira/browse/YARN-4447?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Varun Saxena updated YARN-4447:
-------------------------------
    Attachment: Timeline-Filters.pdf


*1. Metric filters* – Expression for representing metric filters will be as under :
{{(((<expr>[ <op> <expr>…..]) <op> (<expr>[ <op> <expr>…..]))
<op> (<expr>[ <op> <expr>…..]))}}
Here, <op> is a logical operator which can be either “AND” or “OR”.  This will
directly convert to TimelineFilterList$Operator.
Each <expr> would be transformed into single/multiple TimelineCompareFilter objects
and wrapped inside a TimelineFilterList. 
Different <expr> or expressions can be combined together with <op>. Brackets(“(“
and “)”) are used to club together different logical expressions. Square bracket is not
part of the representation and merely denotes that inside within a single opening and closing
bracket, multiple <expr> can be combined using <op>, typically same <op>
.
If brackets are not specified, operators will be parsed left to right with a new TimelineFilterList
created with the old filter list wrapped inside it, whenever <op> changes.

<expr> in turn is of the form :
<key> <compareop> <value>
Here, <key> is metric ID and <value> is the value which will be used to compare
against the metric value.
<key> corresponds to TimelineCompareFilter#key and <value> to TimelineCompareFilter#value.

<compareop> transforms directly to TimelineCompareOp. There are 7 compare ops’ supported.
	gt – Equivalent to TimelineCompareOp.GREATER_THAN
	ge – Equivalent to TimelineCompareOp.GREATER_OR_EQUAL
	lt – Equivalent to TimelineCompareOp.LESS_THAN
	le – Equivalent to TimelineCompareOp.LESS_OR_EQUAL
	eq – Equivalent to TimelineCompareOp.EQUAL
	ne – Equivalent to TimelineCompareOp.NOT_EQUAL. TimelineCompareFilter#keyMustExist will
be set to false. Entity would be returned if key or metric id does not exist.
	ene  – Equivalent to TimelineCompareOp.NOT_EQUAL. TimelineCompareFilter#keyMustExist
will be set to true. Entity would not be returned if key or metric id does not exist.
           _Example:_ 
               {{((metric1 lt 40 OR metric2 gt 80) AND (metric3 eq 10)) OR (metric4 lt 5 AND
metric5 ne 4 OR metric6 ene 7)}}
Please note that all the URL unsafe characters including spaces have to be properly encoded
by client.

*2. Config and Info filters* – Expression for representing config and info filters is exactly
same as metric filters. Only difference being that only 3 <compareop> are supported,
namely, eq, ne and ene. And, the corresponding filter is TimelineKeyValueFilter instead of
TimelineCompareFilter.

*3. Event filters* – Expression for representing event filters will be as under :
{{((\[\!\](<value>\[,<value>,…..\]) <op> \[\!\](<value>\[,<value>,…\]))
<op> \[\!\](<value>\[,<value>,…..\]))}}
Here also, <op> is a logical operator which can be either “AND” or “OR”.  This
will directly convert to TimelineFilterList$Operator.
Each <value> here means event ID and would go to TimelineExistsFilter#value. A comma
separated list of event IDs’ would be transformed into single(if no comma) or multiple TimelineExistsFilter
objects and wrapped inside a TimelineFilterList with Operator AND.

“\!” here means NOT. A comma separated list of events inside an opening and closing bracket
pair with a “!” before opening bracket (”(“) means the events should not exist for
an entity to match. The TimelineCompareOp in each of these TimelineExistsFilter(s) here would
be set to NOT_EQUAL.
If there is no “!” before opening bracket, it means the corresponding TimelineCompareOp
will be  EQUAL.

Basically a sub-expression such as (event1,event2,event3)  or event1,event2,event3 would transform
to a TimelineFilterList with Operator “AND” and containing following filters : 
TimelineExistsFilter (event1, TimelineCompareOp.EQUAL), 
TimelineExistsFilter (event2, TimelineCompareOp.EQUAL)
and 
TimelineExistsFilter (event3, TimelineCompareOp.EQUAL)

If the subexpression is !(event1,event2,event3) would transform to a TimelineFilterList with
Operator “AND” and containing following filters : 
TimelineExistsFilter(event1, TimelineCompareOp.NOT_EQUAL), 
TimelineExistsFilter (event2, TimelineCompareOp.NOT_EQUAL)
and 
TimelineExistsFilter (event3, TimelineCompareOp.NOT_EQUAL)
Please note brackets cannot be omitted if we want compare op to be NOT_EQUAL. The comma separated
list of values(events) must be within brackets with a “!” preceding opening bracket.
    _Example :_  
            {{((((event1,event2,event3) AND !(event4,event5)) OR event6,event7) AND !(event8))}}

*4. Relation filters* – Expression for representing relation filters is same as event filters
except that the value is treated differently. 
Basic expression is same as event filters. But the <value> for relation filters is further
represented as :
{{<entity_type>:<entity_id>\[:<entity_id>:<entity_id>…\]}}
It is a colon separated list of entity type and IDs’. Spaces are not allowed in expression
above. 
The corresponding filter constructed for relation filters is TimelineKeyValuesFilter.  
    _Example :_  
        {{((((type1:entity11:entity12:entity13,type2:entity21,type3:entity31:entity32) AND
 !(type4:entity41,type5:entity51)) OR type6:entity6,type7:entity71) AND !(type2:entity25))}}

*5. MetricsToRetrieve/ConfsToRetrieve* – Expression for representing metrics / configs to
retrieve is as under : 
{{\[\!\](<value>\[,<value>,…..\])}}
Here we have no <op>. “!” has same meaning as event filters i.e. its considered
as compare op of NOT_EQUAL.
If “!” is not specified, compare op will be EQUAL. Please note brackets are not required
if compare op is intended to be EQUAL.
<value> represents a config prefix if expression is for confs to retrieve.
If expression is for metrics to retrieve, <value> represents a metric prefix.
Each value is transformed into TimelinePrefixFilter and then wrapped inside a TimelineFilterList.
If compare op is NOT_EQUAL, TimelineFilterList will be constructed with Operator AND, and
if compare op is EQUAL, TimelineFilterList will be constructed with Operator OR.


> Provide a mechanism to represent complex filters and parse them at the REST layer 
> ----------------------------------------------------------------------------------
>
>                 Key: YARN-4447
>                 URL: https://issues.apache.org/jira/browse/YARN-4447
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Varun Saxena
>            Assignee: Varun Saxena
>              Labels: yarn-2928-1st-milestone
>         Attachments: Timeline-Filters.pdf, YARN-4447-YARN-2928.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message