reef-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Markus Weimer <wei...@apache.org>
Subject Re: Adding LoggingScope for Java REEF
Date Sun, 19 Oct 2014 16:35:17 GMT
Hi,

this looks like a good thing to have in general. We can target various
other aggregation frameworks in the future once our code is littered with
calls to it. A couple of thoughts / questions on the specifics:


   - In REEF, we have only a few things (tasks, contexts, evaluators) we
   can have events on. It might make sense to special-case the API for those.
   That doesn't mean that we shouldn't also allow for arbitrary strings, of
   course.
   - Instead of calling `new` in the try-block, how about using a factory?
   Together with the prior suggestion, we'd have something like
   `try(LoggingScope ls = factory.evaluatorAllocated(evaluatorId)){...}` We
   can then have the factory injected via Tang, which will give us a handle to
   replace the implementations in the future.
   - Some of these timings can be done by REEF. For instance, we should
   wrap all calls to application event handles with this new API. That way, we
   can create generic performance reports for *all* REEF applications without
   them doing any extra work. Given that all such calls are already in
   try-blocks to catch their exceptions, this shouldn't be too hard to do.

Markus

On Sun, Oct 19, 2014 at 1:18 AM, Brian Cho <chobrian@gmail.com> wrote:

> Hi Julia,
>
>
> I'd be interested in using something like this to profile my code. Using
> autocloseable looks appropriate.
>
>
>
>
> It would also be nice if this could be tied into more runtime metrics
> aggregation e.g. something like or built on
> https://dropwizard.github.io/metrics/3.1.0/
>
>
>
>
> This might also be overkill for the situation. Any thoughts?
>
>
>
>
> -Brian
>
> On Sat, Oct 18, 2014 at 9:36 AM, Julia Wang (QIUHE)
> <Qiuhe.Wang@microsoft.com> wrote:
>
> > To measure the performance at function and code block level, I would
> like to add a new class LoggingScope in REEF, similar to what we have in
> .Net version.
> > For any specified code block, the object of the LoggingScope will start
> measure the time until the block  exists. It will log the duration in
> millisecond in the end.
> > This class implements java.lang.AutoCloseable. For the usage, we can
> use  Automatic Resource Management using try keyword for the code block.
> > Example is
> >   final class AllocatedEvaluatorHandler implements
> EventHandler<AllocatedEvaluator> {
> >     @Override
> >     public void onNext(final AllocatedEvaluator allocatedEvaluator) {
> >       try (LoggingScope ls = new LoggingScope(LOG,
> "JobDriver::AllocatedEvaluatorHandler")) {
> >         synchronized (JobDriver.this) {
> >
>  perfLogger.logTime("JobDriver::AllocatedEvaluatorHandler.OnNext");
> >           if (JobDriver.this.nCLREvaluators > 0) {
> >             JobDriver.this.submitEvaluator(allocatedEvaluator,
> EvaluatorType.CLR);
> >             JobDriver.this.nCLREvaluators--;
> >           }
> >         }
> >       }
> > }
> > The result will be logged as:
> > START:JobDriver::AllocatedEvaluatorHandler on Fri Oct 17 17:28:13 PDT
> 2014
> > END:JobDriver::AllocatedEvaluatorHandler on Fri Oct 17 17:28:16 PDT 2014
> Duration=2839
> > The log level can be defined as FINE or FINEST up to what we want. The
> try block can be at code pieces.
> > Let me know if you have feedbacks.
> > Thanks,
> > Julia
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message