Mailing-List: contact dev-help@storm.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@storm.incubator.apache.org
Date: Fri, 21 Feb 2014 06:17:21 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: dev@storm.incubator.apache.org
Message-ID: <JIRA.12691562.1390887161539.79413.1392963441265@arcas>
In-Reply-To: <JIRA.12691562.1390887161539@arcas>
References: <JIRA.12691562.1390887161539@arcas>
Subject: [jira] [Commented] (STORM-200) Proposal for Multilang's Metrics
 feature
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/STORM-200?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D13908=
005#comment-13908005 ]=20

ASF GitHub Bot commented on STORM-200:
--------------------------------------

GitHub user dashengju opened a pull request:

    https://github.com/apache/incubator-storm/pull/38

    STORM-200 Proposal for Multilang's Metrics feature

    for https://issues.apache.org/jira/browse/STORM-200
   =20
    Storm 0.9.0.1 exposes a metrics interface to report summary statistics =
across the full topology. We can build our own metric, and build metrics co=
nsumer to use those statistics.=20
   =20
    But when we use Multilang(ie. Python), we can not use this feature. So =
we want to summit a proposal for multilang's metrics.=20
   =20
    The specifics of the proposal:=20
    1. The main idea is: when user want to add a metric statistics in multi=
lang(python) bolt=EF=BC=8C=20
        a) he need first create a metric object and register in ShellBolt's=
 sub-class,=20
        b) then update the metric in Python bolt process through RPC call.=
=20
   =20
    2. In Metrics API:=20
        a) extend IMetric interface add a method for RPC call=EF=BC=9A
                       public void updateMetricFromRPC(List<Object> params)=
;=20
        b) modify IMetric implements, to support updateMetricFromRPC;=20
   =20
    3. In ShellBolt=EF=BC=8C=20
        a) we have a Map<String, IMetric> to hold user's registered metrics=
 object=EF=BC=9B=20
        b) we have a method registerMetric(String name, T metric) for user =
to register their metrics object;=20
        c) we have a method handleMetrics(Map action) to handle RPC call fr=
om Python process=EF=BC=9B=20
   =20
    4) In Multilang protocol: add a command "metrics" for shell process to =
make RPC call. The protocol is: {"command":"metrics", "name":"metric-regist=
erd-name", "params":["param-1", param-2]}=20
   =20
    5) In storm.py=EF=BC=9Aadd rpcMetrics(name, params), user can update re=
mote metric through this RPC call.=20
   =20
    --------ExamplePythonBolt.java-----------------------------------------=
-----------------------------------------------------------
    public class ExamplePythonBolt extends ShellBolt implements IRichBolt {
    =09private static final long serialVersionUID =3D 1999209252187463355L;
   =20
    =09private TopologyConfig topologyConfig;
    =09
    =09public ExamplePythonBolt(TopologyConfig config) {
    =09=09super("python", "ExampleBolt.py", config.topologyName);
    =09=09topologyConfig =3D config;
    =09}
    =09
        public void prepare(Map stormConf, TopologyContext context, OutputC=
ollector collector) {
        =09super.prepare(stormConf, context, collector);
        =09
        =09CountMetric cMetric =3D new CountMetric();                      =
   //generate a metric object
        =09context.registerMetric("PythonBoltCount", cMetric, 120);  //regi=
ster metric object to context
        =09this.registerMetric("PythonBoltCount", cMetric);                =
//register metric object to ShellBolt
        }
   =20
    =09public void declareOutputFields(OutputFieldsDeclarer declarer) {
    =09=09declarer.declare(new Fields("LogRecord"));
    =09}
   =20
    =09public Map<String, Object> getComponentConfiguration() {
    =09=09return null;
    =09}
    }
   =20
    --------ExampleBolt.py-------------------------------------------------=
--------------------------------------------------
    class ExampleBolt(storm.BasicBolt):
        def __init__(self, boltParams):
            #from here, you can get params from java
            pass
       =20
        def initialize(self, stormconf, context):
            pass
           =20
        def process(self, tup):
            try:        =20
               =20
                MTLOGGER.info("TestAction", "handle the json %s" % (logjson=
)) =20
                storm.emit([tup[0]])
               =20
                storm.rpcMetrics("PythonBoltCount", [1])        //update yo=
ur PythonBoltCount metric with param list "[1]"
            except Exception,tx:
                //handle error


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dashengju/incubator-storm master

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-storm/pull/38.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #38
   =20
----
commit edadde30420e5b2cd3e3d248a6a2268f8dedc22f
Author: JuDasheng <judasheng@meituan.com>
Date:   2014-01-28T06:47:30Z

    modify metrics system to support shellbolt's metrics

commit 52a4fbbd670cfac3a9174bd6c7ad63d3a2282fa8
Author: JuDasheng <judasheng@meituan.com>
Date:   2014-02-21T06:02:28Z

    modify for metrics

----


> Proposal for Multilang's Metrics feature
> ----------------------------------------
>
>                 Key: STORM-200
>                 URL: https://issues.apache.org/jira/browse/STORM-200
>             Project: Apache Storm (Incubating)
>          Issue Type: New Feature
>            Reporter: DashengJu
>            Priority: Minor
>
> Storm 0.9.0.1 exposes a metrics interface to report summary statistics ac=
ross the full topology. We can build our own metric, and build metrics cons=
umer to use those statistics.
> But when we use Multilang(ie. Python), we can not use this feature. So we=
 want to summit a proposal for multilang's metrics.=20
> The specifics of the proposal:
> 1. The main idea is: when user want to add a metric statistics in multila=
ng(python) bolt=EF=BC=8C
>     a) he need first create a metric object and register in ShellBolt's s=
ub-class,=20
>     b) then update the metric in Python bolt process through RPC call.
> 2. In Metrics API:
>     a) extend IMetric interface add a method for RPC call=EF=BC=9Apublic =
void updateMetricFromRPC(List<Object> params); =20
>     b) modify IMetric implements, to support updateMetricFromRPC;
> 3. In ShellBolt=EF=BC=8C
>     a) we have a Map<String, IMetric> to hold user's registered metrics  =
object=EF=BC=9B
>     b) we have a method registerMetric(String name, T metric) for user to=
 register their metris object;
>     c) we have a method handleMetrics(Map action) to handle RPC call from=
 Python process=EF=BC=9B
> 4) In Multilang protocol: add a command "metrics" for shell process to ma=
ke RPC call. The protocol is: {"command":"metrics", "name":"metric-register=
d-name", "params":["param-1", param-2]}
> 5) In storm.py=EF=BC=9Aadd rpcMetrics(name, params), user can update remo=
te metric through this RPC call.
> any suggestions?


--
This message was sent by Atlassian JIRA
(v6.1.5#6160)