incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthieu Morel (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (S4-86) Metrics
Date Mon, 25 Feb 2013 18:20:13 GMT

    [ https://issues.apache.org/jira/browse/S4-86?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13586079#comment-13586079
] 

Matthieu Morel commented on S4-86:
----------------------------------

We added registering of various kinds of statistics about S4 in S4-95, which has already been
merged into dev branch.

In addition, I added a simple mechanism to enable reporting to the console or to csv files,
uploaded to branch S4-86.

The `s4.metrics.config` parameter enables periodic dumps of aggregated statistics to the console
or to files in csv format. This parameter is specified as an application parameter, and must
match the following regular expression: 

	(csv:.+|console):(\d+):(DAYS|HOURS|MICROSECONDS|MILLISECONDS|MINUTES|NANOSECONDS|SECONDS)

Examples:
	
	# dump metrics to csv files to /path/to/directory every 10 seconds
	csv:file://path/to/directory:10:SECONDS
	
	# dump metrics to the console every minute
	console:1:MINUTES

                
> Metrics
> -------
>
>                 Key: S4-86
>                 URL: https://issues.apache.org/jira/browse/S4-86
>             Project: Apache S4
>          Issue Type: New Feature
>    Affects Versions: 0.6
>            Reporter: Matthieu Morel
>             Fix For: 0.6
>
>
> Improving performance of S4 applications requires metrics that can be analyzed.
> Some metrics suggested in S4-65 are:
> * event rate
> * average/median event processing time
> * event queue sizes
> * number of PE instances
> * number of processed events ( maybe within a sliding window)
> * exceptions during event processing
> We'd also add things like:
> * shedding stats
> * memory usage
> * CPU load
> * checkpointing: effective checkpoints, rejected ones
> * etc...
> Ideally, we'd be able to have different kinds of metrics, including historical ones.
> We also want to be able to expose those metrics through JMX, and probably through other
monitoring systems such as nagios.
> A library that could be useful here is codahale's metrics http://metrics.codahale.com/
, but there are alternatives that could be evaluated as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message