hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mostafa Elhemali (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-9090) Refactor MetricsSystemImpl to allow for an on-demand publish system
Date Tue, 27 Nov 2012 18:57:58 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-9090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Mostafa Elhemali updated HADOOP-9090:
-------------------------------------

    Attachment: HADOOP-9090.justEnhanceDefaultImpl.patch

Thanks for the suggestion Luke! I've attached an alternative patch doing what you're suggesting:
adding a publishMetricsNow() method that synchronously publishes all metrics on the same thread
in the default implementation.

The alternative patch is much simpler, and honestly I'm having a "why didn't I think of that?"
moments right now. If people are OK with the change of the default implementation (and the
addition of the new interface method) and I'm not missing any race conditions (I'll keep looking)
then I think the simpler patch would work just fine for my purposes.
                
> Refactor MetricsSystemImpl to allow for an on-demand publish system
> -------------------------------------------------------------------
>
>                 Key: HADOOP-9090
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9090
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: metrics
>            Reporter: Mostafa Elhemali
>            Priority: Minor
>         Attachments: HADOOP-9090.2.patch, HADOOP-9090.justEnhanceDefaultImpl.patch, HADOOP-9090.patch
>
>
> We have a need to publish metrics out of some short-living processes, which is not really
well-suited to the current metrics system implementation which periodically publishes metrics
asynchronously (a behavior that works great for long-living processes). Of course I could
write my own metrics system, but it seems like such a waste to rewrite all the awesome code
currently in the MetricsSystemImpl and supporting classes.
> The way I'm proposing to solve this is to:
> 1. Refactor the MetricsSystemImpl class into an abstract base MetricsSystemImpl class
(common configuration and other code) and a concrete PeriodicPublishMetricsSystemImpl class
(timer thread).
> 2. Refactor the MetricsSinkAdapter class into an abstract base MetricsSinkAdapter class
(common configuration and other code) and a concrete AsyncMetricsSinkAdapter class (asynchronous
publishing using the SinkQueue).
> 3. Derive a new simple class OnDemandPublishMetricsSystemImpl from MetricsSystemImpl,
that just exposes a synchronous publish() method to do all the work.
> 4. Derive a SyncMetricsSinkAdapter class from MetricsSinkAdapter to just synchronously
push metrics to the underlying sink.
> Does that sound reasonable? I'll attach the patch with all this coded up and simple tests
(could use some polish I guess, but wanted to get everyone's opinion first). Notice that this
is somewhat of a breaking change since MetricsSystemImpl is public (although it's marked with
InterfaceAudience.Private); if the breaking change is a problem I could just rename the refactored
classes so that PeriodicPublishMetricsSystemImpl is still called MetricsSystemImpl (and MetricsSystemImpl
-> BaseMetricsSystemImpl).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message