hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mostafa Elhemali <mostafa.elhem...@gmail.com>
Subject Re: Refactor MetricsSystemImpl to allow for an on-demand publish system (HADOOP-9090)
Date Tue, 27 Nov 2012 18:22:02 GMT
Thanks for the feedback Luke. You make a good case - to be honest I shyed
away from modifying the default implementation since I figured it's better
to leave existing functionality as is, but you're right that being forced
to choose an implementation (on-demand vs. periodic) can get annoying. Let
me try out that approach and either post an alternative patch to the JIRA
or report back on why I failed.

On Tue, Nov 27, 2012 at 9:37 AM, Luke Lu <llu@vicaya.com> wrote:

> Thanks for the heads up Mostafa. I've added commment/questions on the JIRA.
> The main concern is that now you have to decide what metrics system to use
> depending on the process is long running or not. Why not just enhance the
> existing metrics system itself to support on-demand publishing? So that you
> can use the same metrics system for all processes? We all know a
> short-lived process can live longer than expected and periodic metrics from
> that process can provide insights on what's going on, which is the main
> purpose of the metrics system.
> __Luke
> On Mon, Nov 26, 2012 at 7:41 PM, Mostafa Elhemali <
> mostafa.elhemaly@gmail.com> wrote:
> > Hi all,
> > Yesterday I filed a JIRA
> > (HADOOP-9090<https://issues.apache.org/jira/browse/HADOOP-9090>)
> > to propose a refactoring of the MetricsSystemImpl class - the default
> > (only?) implementation of the Metrics2 system - to factor out some common
> > code in a base class and have another simple implementation that just
> does
> > on-demand publishing of metrics instead of the default periodic
> publishing.
> > The main motivation for filing this JIRA and the attached patch is that
> we
> > (Microsoft) have a need to publish metrics out of short-lived processes
> > (think "hadoop fs -ls") and the periodic behavior of the default
> > implementation doesn't really work well for those. We could write our own
> > metrics system implementation (and we'll probably do that in the short
> > term) but that would mean duplicating a lot of great code that's already
> in
> > the MetricsSystemImpl class, hence the proposal to factor out the common
> > code into a base class.
> >
> > Does that sound reasonable? Please comment on the JIRA directly or reply
> > here - if the proposal sounds awful (or great) or there's something I'm
> > fundamentally missing I'd love to hear that feedback.
> >
> > Thanks!
> > Mostafa
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message