ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dmitry Lysnichenko (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMBARI-4881) Clean up JMXPropertyProvider hacks for STORM metrics
Date Fri, 28 Feb 2014 19:06:20 GMT

     [ https://issues.apache.org/jira/browse/AMBARI-4881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Dmitry Lysnichenko updated AMBARI-4881:
---------------------------------------

    Description: 
h2. The task
Clean up a JMXPropertyProvider hacks for STORM metrics
Also,
- probably, create a separate provider for rest api 
- current implementation is only addressing /api/cluster/summary from Storm.  Allow accessing
other urls like  /api/supervisors/summary etc
- When we refactor (assuming using the HttpProxyPropertyProvider), make that be able to take
any type of reader.  Also, we want to move to Gson, not Jackson.
- Is it possible to set the port of the STORM_REST_API component using the UI?  If so, then
this will result in a failed query.

h3. Create RestPropertyProvider
RestPropertyProvider implementation will be registered at StackDefinedPropertyProvider like
it is done for JMXPropertyProvider and GangliaPropertyProvider. Also I'm going to reuse JMXHostProvider
for resolving component hostsnames. 

h3. Add new metrics type "rest" 
{code}
{
  "type": "rest",
  "properties": {
     "default_port": "8745" # Storm port is always hardcoded, so we will just use the value
     "relevant_config": "storm-site" # Just example, not needed for Storm
     "port_property": "storm.port" # Just example, not needed for Storm
  }
  "metrics": {
    "metrics/api/cluster/summary/nimbus.uptime": {
      "metric": "nimbus.uptime",
      "pointInTime": true,
      "temporal": true
    },
   ...
{code}
Metric group properties define port-related settings. If "port_property" and "relevant_config"
keys are defined, provider will try to extract port setting from the service config. Otherwise,
provider will use the default port. RestPropertyProvider maintains a cache of per-component
port numbers just like we do at org.apache.ambari.server.controller.internal.AbstractProviderModule#getPort
Metric id "metrics/api/cluster/summary/nimbus.uptime" contains both url and property name.
We can not use sole url as metric id because defining few metrics with the same id is not
possible.

h3. Adding to RestPropertyProvider support of nested json properties.
Given a json file like:
{code}
{
  "a" : "b",
  "c" : {
    "d" : "e",
  }
}
{code}
We may get "e" value by defining metric in form metrics/api/cluster/summary/c#d where "api/cluster/summary"
is a json url, and "c#d" points to property "d" nested under property "c". Any nesting depth
is supported. Accessing properties inside json value lists is not supported (I don't see any
use case for that).
Going this way, we will be able to access metrics defined at (almost) arbitrary json files
at arbitrary ports/urls. 

  was:
h2. The task
Clean up a JMXPropertyProvider hacks for STORM metrics
Also,
- probably, create a separate provider for rest api 
- current implementation is only addressing /api/cluster/summary from Storm.  Allow accessing
other urls like  /api/supervisors/summary etc
- When we refactor (assuming using the HttpProxyPropertyProvider), make that be able to take
any type of reader.  Also, we want to move to Gson, not Jackson.
- Is it possible to set the port of the STORM_REST_API component using the UI?  If so, then
this will result in a failed query.

h3. Create RestPropertyProvider
RestPropertyProvider implementation will be registered at StackDefinedPropertyProvider like
it is done for JMXPropertyProvider and GangliaPropertyProvider. Also I'm going to reuse JMXHostProvider
for resolving component hostsnames. 

h3. Add new metrics type "rest" 
{code}
{
  "type": "rest",
  "properties": {
     "default_port": "60010"
     "relevant_config": "storm-site" # Just example, not needed for Storm
     "port_property": "storm.port" # Just example, not needed for Storm
  }
  "metrics": {
    "metrics/api/cluster/summary/nimbus.uptime": {
      "metric": "nimbus.uptime",
      "pointInTime": true,
      "temporal": true
    },
   ...
{code}
Metric group properties define port-related settings. If "port_property" and "relevant_config"
keys are defined, provider will try to extract port setting from the service config. Otherwise,
provider will use the default port. RestPropertyProvider maintains a cache of per-component
port numbers just like we do at org.apache.ambari.server.controller.internal.AbstractProviderModule#getPort
Metric id "metrics/api/cluster/summary/nimbus.uptime" contains both url and property name.
We can not use sole url as metric id because defining few metrics with the same id is not
possible.

h3. Adding to RestPropertyProvider support of nested json properties.
Given a json file like:
{code}
{
  "a" : "b",
  "c" : {
    "d" : "e",
  }
}
{code}
We may get "e" value by defining metric in form metrics/api/cluster/summary/c#d where "api/cluster/summary"
is a json url, and "c#d" points to property "d" nested under property "c". Any nesting depth
is supported. Accessing properties inside json value lists is not supported (I don't see any
use case for that).
Going this way, we will be able to access metrics defined at (almost) arbitrary json files
at arbitrary ports/urls. 


> Clean up JMXPropertyProvider hacks for STORM metrics
> ----------------------------------------------------
>
>                 Key: AMBARI-4881
>                 URL: https://issues.apache.org/jira/browse/AMBARI-4881
>             Project: Ambari
>          Issue Type: Task
>          Components: controller, test
>    Affects Versions: 1.5.0
>            Reporter: Dmitry Lysnichenko
>            Assignee: Dmitry Lysnichenko
>             Fix For: 1.5.0
>
>
> h2. The task
> Clean up a JMXPropertyProvider hacks for STORM metrics
> Also,
> - probably, create a separate provider for rest api 
> - current implementation is only addressing /api/cluster/summary from Storm.  Allow accessing
other urls like  /api/supervisors/summary etc
> - When we refactor (assuming using the HttpProxyPropertyProvider), make that be able
to take any type of reader.  Also, we want to move to Gson, not Jackson.
> - Is it possible to set the port of the STORM_REST_API component using the UI?  If so,
then this will result in a failed query.
> h3. Create RestPropertyProvider
> RestPropertyProvider implementation will be registered at StackDefinedPropertyProvider
like it is done for JMXPropertyProvider and GangliaPropertyProvider. Also I'm going to reuse
JMXHostProvider for resolving component hostsnames. 
> h3. Add new metrics type "rest" 
> {code}
> {
>   "type": "rest",
>   "properties": {
>      "default_port": "8745" # Storm port is always hardcoded, so we will just use the
value
>      "relevant_config": "storm-site" # Just example, not needed for Storm
>      "port_property": "storm.port" # Just example, not needed for Storm
>   }
>   "metrics": {
>     "metrics/api/cluster/summary/nimbus.uptime": {
>       "metric": "nimbus.uptime",
>       "pointInTime": true,
>       "temporal": true
>     },
>    ...
> {code}
> Metric group properties define port-related settings. If "port_property" and "relevant_config"
keys are defined, provider will try to extract port setting from the service config. Otherwise,
provider will use the default port. RestPropertyProvider maintains a cache of per-component
port numbers just like we do at org.apache.ambari.server.controller.internal.AbstractProviderModule#getPort
> Metric id "metrics/api/cluster/summary/nimbus.uptime" contains both url and property
name. We can not use sole url as metric id because defining few metrics with the same id is
not possible.
> h3. Adding to RestPropertyProvider support of nested json properties.
> Given a json file like:
> {code}
> {
>   "a" : "b",
>   "c" : {
>     "d" : "e",
>   }
> }
> {code}
> We may get "e" value by defining metric in form metrics/api/cluster/summary/c#d where
"api/cluster/summary" is a json url, and "c#d" points to property "d" nested under property
"c". Any nesting depth is supported. Accessing properties inside json value lists is not supported
(I don't see any use case for that).
> Going this way, we will be able to access metrics defined at (almost) arbitrary json
files at arbitrary ports/urls. 



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message