chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Graham (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CHUKWA-515) Statistics REST API for Agent
Date Tue, 31 Aug 2010 18:26:52 GMT

    [ https://issues.apache.org/jira/browse/CHUKWA-515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12904707#action_12904707
] 

Bill Graham commented on CHUKWA-515:
------------------------------------

I'm thinking that the {{/adaptorStats}} functionality can be merged into the {{/adaptor}}
REST resource and we just show the stats with the adaptor. I've implemented the REST GET functionality
for all adaptors and a single adaptor by id. The former is shown below (xml and text). The
latter would look similar, just without the {{<Adaptors>}} element.

{noformat}
<Response>
  <Adaptors total="1">
    <Adaptor id="adaptor_c14aa68e64bf0f12be76ce91e7f2e20d"
             dataType="some-data-type" offset="51394422">
      <AdaptorClass>
        org.apache.hadoop.chukwa.datacollection.adaptor.jms.JMSAdaptor
      </AdaptorClass>
      <AdaptorParams>
        some-data-type tcp://jms.host.com:61616 -t queue.name
      </AdaptorParams>
      <AverageRate intervalSeconds="60">17784.66</AverageRate>
      <AverageRate intervalSeconds="300">17733.91</AverageRate>
      <AverageRate intervalSeconds="600">17679.28</AverageRate>
    </Adaptor>
  </Adaptors>
</Response>
{noformat}

For the text view, I'm using YAML:

{noformat}
adaptor_count: 1
adaptors: 
  - adaptor_id: adaptor_c14aa68e64bf0f12be76ce91e7f2e20d
    data_type: some-data-type
    offset: 51355632
    adaptor_class: org.apache.hadoop.chukwa.datacollection.adaptor.jms.JMSAdaptor
    adaptor_params: some-data-type tcp://jms.host.com:61616 -t queue.name
    average_rates: 
      - rate: 17784.66
        interval: 60
      - rate: 17733.91
        interval: 300
      - rate: 17679.28
        interval: 600
{noformat}

I implemented a timer that takes snapshots every 10 seconds and saves up to 15 minutes worth
of data per adaptor. To compute stats I have a {{StatsManager}} that requires a recent data
point within 0.25*interval (to assure data is not stale) and an older data point within 0.25*interval
of the recent data point - interval (to assure adequate history).

Comments?

> Statistics REST API for Agent
> -----------------------------
>
>                 Key: CHUKWA-515
>                 URL: https://issues.apache.org/jira/browse/CHUKWA-515
>             Project: Chukwa
>          Issue Type: Sub-task
>            Reporter: Bill Graham
>            Assignee: Bill Graham
>
> Develop a REST interface to expose Agent statistics. 
> * Request URI:
> {noformat}
> GET /rest/v1/adaptorStats HTTP/1.0
> {noformat}
> * For now I'm shooting for the XML response, but ultimately we can support optional params:
> {noformat}
> viewType=[json|xml|text] (default is XML?)
> {noformat}
> * Response XML
> TBD. My thought is to integrate with the {{AgentMetrics}} and {{ChunkQueueMetrics}} and
show their information, along with a list of adaptors. It appears that the same {{MemQueue}}
is shared amongst all adaptors, so we won't be able to expose per-adaptor stats as is. Let
me know if I'm missing something there, or if you have comments in general about the approach.
> I was planning on using Jetty similar to how the collector does. We could have some common
code that delegates requests to different handlers based on the URI. The current telnet interface
will remain as-is.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message