Return-Path: X-Original-To: apmail-ambari-dev-archive@www.apache.org Delivered-To: apmail-ambari-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B7280106D7 for ; Fri, 28 Feb 2014 19:06:27 +0000 (UTC) Received: (qmail 7144 invoked by uid 500); 28 Feb 2014 19:06:22 -0000 Delivered-To: apmail-ambari-dev-archive@ambari.apache.org Received: (qmail 6945 invoked by uid 500); 28 Feb 2014 19:06:21 -0000 Mailing-List: contact dev-help@ambari.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@ambari.apache.org Delivered-To: mailing list dev@ambari.apache.org Received: (qmail 6805 invoked by uid 99); 28 Feb 2014 19:06:20 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 28 Feb 2014 19:06:20 +0000 Date: Fri, 28 Feb 2014 19:06:20 +0000 (UTC) From: "Dmitry Lysnichenko (JIRA)" To: dev@ambari.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (AMBARI-4881) Clean up JMXPropertyProvider hacks for STORM metrics MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/AMBARI-4881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitry Lysnichenko updated AMBARI-4881: --------------------------------------- Description: h2. The task Clean up a JMXPropertyProvider hacks for STORM metrics Also, - probably, create a separate provider for rest api - current implementation is only addressing /api/cluster/summary from Storm. Allow accessing other urls like /api/supervisors/summary etc - When we refactor (assuming using the HttpProxyPropertyProvider), make that be able to take any type of reader. Also, we want to move to Gson, not Jackson. - Is it possible to set the port of the STORM_REST_API component using the UI? If so, then this will result in a failed query. h3. Create RestPropertyProvider RestPropertyProvider implementation will be registered at StackDefinedPropertyProvider like it is done for JMXPropertyProvider and GangliaPropertyProvider. Also I'm going to reuse JMXHostProvider for resolving component hostsnames. h3. Add new metrics type "rest" {code} { "type": "rest", "properties": { "default_port": "8745" # Storm port is always hardcoded, so we will just use the value "relevant_config": "storm-site" # Just example, not needed for Storm "port_property": "storm.port" # Just example, not needed for Storm } "metrics": { "metrics/api/cluster/summary/nimbus.uptime": { "metric": "nimbus.uptime", "pointInTime": true, "temporal": true }, ... {code} Metric group properties define port-related settings. If "port_property" and "relevant_config" keys are defined, provider will try to extract port setting from the service config. Otherwise, provider will use the default port. RestPropertyProvider maintains a cache of per-component port numbers just like we do at org.apache.ambari.server.controller.internal.AbstractProviderModule#getPort Metric id "metrics/api/cluster/summary/nimbus.uptime" contains both url and property name. We can not use sole url as metric id because defining few metrics with the same id is not possible. h3. Adding to RestPropertyProvider support of nested json properties. Given a json file like: {code} { "a" : "b", "c" : { "d" : "e", } } {code} We may get "e" value by defining metric in form metrics/api/cluster/summary/c#d where "api/cluster/summary" is a json url, and "c#d" points to property "d" nested under property "c". Any nesting depth is supported. Accessing properties inside json value lists is not supported (I don't see any use case for that). Going this way, we will be able to access metrics defined at (almost) arbitrary json files at arbitrary ports/urls. was: h2. The task Clean up a JMXPropertyProvider hacks for STORM metrics Also, - probably, create a separate provider for rest api - current implementation is only addressing /api/cluster/summary from Storm. Allow accessing other urls like /api/supervisors/summary etc - When we refactor (assuming using the HttpProxyPropertyProvider), make that be able to take any type of reader. Also, we want to move to Gson, not Jackson. - Is it possible to set the port of the STORM_REST_API component using the UI? If so, then this will result in a failed query. h3. Create RestPropertyProvider RestPropertyProvider implementation will be registered at StackDefinedPropertyProvider like it is done for JMXPropertyProvider and GangliaPropertyProvider. Also I'm going to reuse JMXHostProvider for resolving component hostsnames. h3. Add new metrics type "rest" {code} { "type": "rest", "properties": { "default_port": "60010" "relevant_config": "storm-site" # Just example, not needed for Storm "port_property": "storm.port" # Just example, not needed for Storm } "metrics": { "metrics/api/cluster/summary/nimbus.uptime": { "metric": "nimbus.uptime", "pointInTime": true, "temporal": true }, ... {code} Metric group properties define port-related settings. If "port_property" and "relevant_config" keys are defined, provider will try to extract port setting from the service config. Otherwise, provider will use the default port. RestPropertyProvider maintains a cache of per-component port numbers just like we do at org.apache.ambari.server.controller.internal.AbstractProviderModule#getPort Metric id "metrics/api/cluster/summary/nimbus.uptime" contains both url and property name. We can not use sole url as metric id because defining few metrics with the same id is not possible. h3. Adding to RestPropertyProvider support of nested json properties. Given a json file like: {code} { "a" : "b", "c" : { "d" : "e", } } {code} We may get "e" value by defining metric in form metrics/api/cluster/summary/c#d where "api/cluster/summary" is a json url, and "c#d" points to property "d" nested under property "c". Any nesting depth is supported. Accessing properties inside json value lists is not supported (I don't see any use case for that). Going this way, we will be able to access metrics defined at (almost) arbitrary json files at arbitrary ports/urls. > Clean up JMXPropertyProvider hacks for STORM metrics > ---------------------------------------------------- > > Key: AMBARI-4881 > URL: https://issues.apache.org/jira/browse/AMBARI-4881 > Project: Ambari > Issue Type: Task > Components: controller, test > Affects Versions: 1.5.0 > Reporter: Dmitry Lysnichenko > Assignee: Dmitry Lysnichenko > Fix For: 1.5.0 > > > h2. The task > Clean up a JMXPropertyProvider hacks for STORM metrics > Also, > - probably, create a separate provider for rest api > - current implementation is only addressing /api/cluster/summary from Storm. Allow accessing other urls like /api/supervisors/summary etc > - When we refactor (assuming using the HttpProxyPropertyProvider), make that be able to take any type of reader. Also, we want to move to Gson, not Jackson. > - Is it possible to set the port of the STORM_REST_API component using the UI? If so, then this will result in a failed query. > h3. Create RestPropertyProvider > RestPropertyProvider implementation will be registered at StackDefinedPropertyProvider like it is done for JMXPropertyProvider and GangliaPropertyProvider. Also I'm going to reuse JMXHostProvider for resolving component hostsnames. > h3. Add new metrics type "rest" > {code} > { > "type": "rest", > "properties": { > "default_port": "8745" # Storm port is always hardcoded, so we will just use the value > "relevant_config": "storm-site" # Just example, not needed for Storm > "port_property": "storm.port" # Just example, not needed for Storm > } > "metrics": { > "metrics/api/cluster/summary/nimbus.uptime": { > "metric": "nimbus.uptime", > "pointInTime": true, > "temporal": true > }, > ... > {code} > Metric group properties define port-related settings. If "port_property" and "relevant_config" keys are defined, provider will try to extract port setting from the service config. Otherwise, provider will use the default port. RestPropertyProvider maintains a cache of per-component port numbers just like we do at org.apache.ambari.server.controller.internal.AbstractProviderModule#getPort > Metric id "metrics/api/cluster/summary/nimbus.uptime" contains both url and property name. We can not use sole url as metric id because defining few metrics with the same id is not possible. > h3. Adding to RestPropertyProvider support of nested json properties. > Given a json file like: > {code} > { > "a" : "b", > "c" : { > "d" : "e", > } > } > {code} > We may get "e" value by defining metric in form metrics/api/cluster/summary/c#d where "api/cluster/summary" is a json url, and "c#d" points to property "d" nested under property "c". Any nesting depth is supported. Accessing properties inside json value lists is not supported (I don't see any use case for that). > Going this way, we will be able to access metrics defined at (almost) arbitrary json files at arbitrary ports/urls. -- This message was sent by Atlassian JIRA (v6.1.5#6160)