hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Luke Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9090) Refactor MetricsSystemImpl to allow for an on-demand publish system
Date Thu, 29 Nov 2012 18:40:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13506654#comment-13506654
] 

Luke Lu commented on HADOOP-9090:
---------------------------------

Adding a publishMetricsNow method to the MetricsSystem is reasonable, as the interface is
considered Evolving and the requirement has universal utility (I actually thought about adding
it in the beginning but there was no such requirement then).

bq. I figured the way you had it may end up in race conditions if multiple threads are calling
publishMetricsNow() at the same time.

The _sketch_ was meant to be simple and the race is considered harmless: it's ok to potentially
exit before one of the metrics buffer that's almost the same time with the last one is flushed.
OTOH, if you want to wait for individual metrics buffer you can do the following without a
new wrapper:
{code}
// in putMetricsImediate
synchronized(buffer) {
  buffer.wait(oobTimeout);
}

// in consume
synchronized(buffer) {
  buffer.notify();
}
{code}
                
> Refactor MetricsSystemImpl to allow for an on-demand publish system
> -------------------------------------------------------------------
>
>                 Key: HADOOP-9090
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9090
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: metrics
>            Reporter: Mostafa Elhemali
>            Priority: Minor
>         Attachments: HADOOP-9090.2.patch, HADOOP-9090.justEnhanceDefaultImpl.2.patch,
HADOOP-9090.justEnhanceDefaultImpl.3.patch, HADOOP-9090.justEnhanceDefaultImpl.4.patch, HADOOP-9090.justEnhanceDefaultImpl.5.patch,
HADOOP-9090.justEnhanceDefaultImpl.patch, HADOOP-9090.patch
>
>
> Updated description based on feedback:
> We have a need to publish metrics out of some short-living processes, which is not really
well-suited to the current metrics system implementation which periodically publishes metrics
asynchronously (a behavior that works great for long-living processes). Of course I could
write my own metrics system, but it seems like such a waste to rewrite all the awesome code
currently in the MetricsSystemImpl and supporting classes.
> The way this JIRA solves this problem is adding a new method publishMetricsNow() to the
MetricsSystemImpl() class, that does a synchronous out-of-band push of the metrics from the
sources to the sink. I also add a method to MetricsSinkAdapter (putMetricsImmediate) to support
that change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message