giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Reisman (JIRA)" <j...@apache.org>
Subject [jira] [Created] (GIRAPH-568) Giraph on YARN will need a WebUI to display job stats, Yammer metrics, whatever
Date Sat, 16 Mar 2013 18:54:12 GMT
Eli Reisman created GIRAPH-568:
----------------------------------

             Summary: Giraph on YARN will need a WebUI to display job stats, Yammer metrics,
whatever
                 Key: GIRAPH-568
                 URL: https://issues.apache.org/jira/browse/GIRAPH-568
             Project: Giraph
          Issue Type: Improvement
    Affects Versions: 0.2.0
            Reporter: Eli Reisman
             Fix For: 0.2.0


In YARN, the Client is the driver program when you run a job at the commmand line. This lauches
the application master, which is like an uber-master that manages the job lifecycle for all
the Giraph worker/master tasks that actually run the BSP job.

The Application Master can register an RPC Port and a Tracking URL with the YARN system (ResourceManager)
which will be published on the YARN cluster WebUI in case folks running a Giraph job want
to see detailed formatted web info such as Hadoop has. Previously we have hijacked Hadoop's
counters and web ui. Now, we can start to think fresh about how to read logs, view job and
node status, memory use, disk spills, Yammer metrics, whatever.

Someone could get very creative with this. If someone is feeling up to it, I can show you
where the YARN bits are you will want to interface with. The rest can really go any way you
want it to.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message