spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gerard Maas <gerard.m...@gmail.com>
Subject Re: Registering custom metrics
Date Mon, 05 Jan 2015 10:24:48 GMT
Hi,

Yes, I managed to create a register custom metrics by creating an
 implementation  of org.apache.spark.metrics.source.Source and registering
it to the metrics subsystem.
Source is [Spark] private, so you need to create it under a org.apache.spark
package. In my case, I'm dealing with Spark Streaming metrics, and I
created my CustomStreamingSource under org.apache.spark.streaming as I also
needed access to some [Streaming] private components.

Then, you register your new metric Source on the Spark's metric system,
like so:

SparkEnv.get.metricsSystem.registerSource(customStreamingSource)

And it will get reported to the metrics Sync active on your system. By
default, you can access them through the metric endpoint:
http://<driver-host>:<ui-port>/metrics/json

I hope this helps.

-kr, Gerard.






On Tue, Dec 30, 2014 at 3:32 PM, eshioji <eshioji@gmail.com> wrote:

> Hi,
>
> Did you find a way to do this / working on this?
> Am trying to find a way to do this as well, but haven't been able to find a
> way.
>
>
>
> --
> View this message in context:
> http://apache-spark-developers-list.1001551.n3.nabble.com/Registering-custom-metrics-tp9030p9968.html
> Sent from the Apache Spark Developers List mailing list archive at
> Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@spark.apache.org
> For additional commands, e-mail: dev-help@spark.apache.org
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message