giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nitay Joffe (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (GIRAPH-232) Add metrics system into Giraph
Date Mon, 15 Oct 2012 07:20:03 GMT

     [ https://issues.apache.org/jira/browse/GIRAPH-232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nitay Joffe updated GIRAPH-232:
-------------------------------

    Attachment: GIRAPH-232.patch

https://reviews.apache.org/r/7595/

Here's my attempt of bringing this up to date and adding some metrics.

Sample output here: https://gist.github.com/3891129

Some info about the Metrics library: http://metrics.codahale.com/manual/core/

Let me know what you guys think. Other metrics you think we should to add, use different types,
etc. I was thinking of also using different groups instead of the class names like "Superstep",
"Network", "System", and so on.

It prints to stdout every 90 seconds (configurable).

This patch fails checkstyle because of all of the @see tags in GiraphMetrics, but I don't
see why these should be errors?

The entire metrics can be toggled on/off but passing -Dgiraph.metrics.enable. To implement
this I created my own set of metrics classes (Timer, Gauge, Metric, and so on) which do nothing
and switch those in if the flag disables all metrics. This way all of the user code stays
untouched yet does nothing. I am chatting with metrics folks about contributing this back
to them.

I was looking for an easy way to disable metrics in tests but from what I could tell our tests
don't really have ONE way of running a job that everyone uses. Some use InternalVertexRunner,
some use BspCase, some create GiraphJob on their own. Seems to me this should all be consolidated
so all test jobs go through some known code path which sets "test mode" things?
                
> Add metrics system into Giraph
> ------------------------------
>
>                 Key: GIRAPH-232
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-232
>             Project: Giraph
>          Issue Type: New Feature
>            Reporter: Jakob Homan
>            Assignee: Jakob Homan
>         Attachments: GIRAPH-231.patch, GIRAPH-232.patch
>
>
> Currently a lot of Giraph's operations are not transparent. As a Hadoop job, the Giraph
logging is at the mercy of Hadoop's logging system and can disappear when one encounters a
memory issue.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message