Mailing-List: contact dev-help@giraph.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@giraph.apache.org
Date: Mon, 15 Oct 2012 07:20:03 +0000 (UTC)
From: "Nitay Joffe (JIRA)" <jira@apache.org>
To: giraph-dev@incubator.apache.org
Message-ID: <564452336.44536.1350285603836.JavaMail.jiratomcat@arcas>
In-Reply-To: <1752786134.419.1341273297527.JavaMail.jiratomcat@issues-vm>
Subject: [jira] [Updated] (GIRAPH-232) Add metrics system into Giraph
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/GIRAPH-232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Nitay Joffe updated GIRAPH-232:
-------------------------------

    Attachment: GIRAPH-232.patch

https://reviews.apache.org/r/7595/

Here's my attempt of bringing this up to date and adding some metrics.

Sample output here: https://gist.github.com/3891129

Some info about the Metrics library: http://metrics.codahale.com/manual/core/

Let me know what you guys think. Other metrics you think we should to add, use different types, etc. I was thinking of also using different groups instead of the class names like "Superstep", "Network", "System", and so on.

It prints to stdout every 90 seconds (configurable).

This patch fails checkstyle because of all of the @see tags in GiraphMetrics, but I don't see why these should be errors?

The entire metrics can be toggled on/off but passing -Dgiraph.metrics.enable. To implement this I created my own set of metrics classes (Timer, Gauge, Metric, and so on) which do nothing and switch those in if the flag disables all metrics. This way all of the user code stays untouched yet does nothing. I am chatting with metrics folks about contributing this back to them.

I was looking for an easy way to disable metrics in tests but from what I could tell our tests don't really have ONE way of running a job that everyone uses. Some use InternalVertexRunner, some use BspCase, some create GiraphJob on their own. Seems to me this should all be consolidated so all test jobs go through some known code path which sets "test mode" things?
                
> Add metrics system into Giraph
> ------------------------------
>
>                 Key: GIRAPH-232
>                 URL: https://issues.apache.org/jira/browse/GIRAPH-232
>             Project: Giraph
>          Issue Type: New Feature
>            Reporter: Jakob Homan
>            Assignee: Jakob Homan
>         Attachments: GIRAPH-231.patch, GIRAPH-232.patch
>
>
> Currently a lot of Giraph's operations are not transparent. As a Hadoop job, the Giraph logging is at the mercy of Hadoop's logging system and can disappear when one encounters a memory issue.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira