accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <josh.el...@gmail.com>
Subject Re: Problem configuring Ganglia(jmxtrans) with Accumulo 1.7.0
Date Wed, 27 Jan 2016 16:20:05 GMT
Mohit,

I would strongly recommend you take a look at the Hadoop Metrics2 
support that was added in 1.7.0 to push metrics to Ganglia instead of 
using JMXTrans.

This was one of the big reasons we chose to adopt the new metrics 
system. Using this integration will prevent you from having to 
run/monitor jmxtrans (which is always nice in a production environment). 
There should be a commented out example of how to configure Ganglia in 
the provided hadoop-metrics2-accumulo.properties file in the example 
configurations.

http://accumulo.apache.org/1.7/accumulo_user_manual.html#_metrics2_configuration

- Josh

mohit.kaushik wrote:
> I am trying to setup Ganglia for Monitoring Accumulo 1.7.0 Cluster
> having 5 nodes. I also configured ganglia for Accumulo 1.6.1 standalone
> cluster which is working fine and shows all the required metrices
> related to Accumulo.
>
> After scanning jmxtrans.log, I found that the its not exexuting quring
> properly. It starts the job and finish in next line
> [27 Jan 2016 17:38:15] [ServerScheduler_Worker-1] 3960536 DEBUG
> (com.googlecode.jmxtrans.jobs.ServerJob:31) - +++++ Started server job:
> Server [host=localhost, port=9006,
> url=service:jmx:rmi:///jndi/rmi://localhost:9006/jmxrmi,
> cronExpression=null, numQueryThreads=null]
> [27 Jan 2016 17:38:15] [ServerScheduler_Worker-1] 3960542 DEBUG
> (com.googlecode.jmxtrans.jobs.ServerJob:50) - +++++ Finished server job:
> Server [host=localhost, port=9006,
> url=service:jmx:rmi:///jndi/rmi://localhost:9006/jmxrmi,
> cronExpression=null, numQueryThreads=null]
>
> But this not the case with Accumulo 1.6.1 cluster it executes the query
> and send the ganglia metrices which I can see on ganglia web ui...
>
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360736 DEBUG
> (com.googlecode.jmxtrans.jobs.ServerJob:31) - +++++ Started server job:
> Server [host=mohit, port=9006,
> url=service:jmx:rmi:///jndi/rmi://mohit:9006/jmxrmi,
> cronExpression=null, numQueryThreads=null]
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360742 DEBUG
> (com.googlecode.jmxtrans.util.JmxUtils:195) - Executing queryName:
> accumulo.server.metrics:instance=tserver,name=TabletServerMBean,service=TServerInfo
> from query: Query
> [obj=accumulo.server.metrics:service=TServerInfo,name=TabletServerMBean,instance=tserver,
> resultAlias=Accumulo, attr=[Entries, EntriesInMemory, Ingest,
> MajorCompactions, MajorCompactionsQueued, MinorCompactions,
> MinorCompactionsQueued, OnlineCount, OpeningCount, Queries,
> UnopenedCount, TotalMinorCompactions, HoldTime, AverageFilesPerTablet,
> Name]]
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360746 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.Entries=14
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360746 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.EntriesInMemory=0
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360747 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.Ingest=0
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360748 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.MajorCompactions=0
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360748 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.MajorCompactionsQueued=0
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360749 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.MinorCompactions=0
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360749 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.MinorCompactionsQueued=0
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360750 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.OnlineCount=3
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360750 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.OpeningCount=0
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360751 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.Queries=995
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360751 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.UnopenedCount=0
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360752 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.TotalMinorCompactions=33
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360752 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.HoldTime=0.0
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360752 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.AverageFilesPerTablet=0.3333333333333333
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360753 DEBUG
> (com.googlecode.jmxtrans.model.output.GangliaWriter:141) - Sending
> Ganglia metric Accumulo.Name=mohit:9997
> [27 Jan 2016 17:39:25] [ServerScheduler_Worker-9] 360753 DEBUG
> (com.googlecode.jmxtrans.util.JmxUtils:209) - Finished running
> outputWriters for query: Query
> [obj=accumulo.server.metrics:service=TServerInfo,name=TabletServerMBean,instance=tserver,
> resultAlias=Accumulo, attr=[Entries, EntriesInMemory, Ingest,
> MajorCompactions, MajorCompactionsQueued, MinorCompactions,
> MinorCompactionsQueued, OnlineCount, OpeningCount, Queries,
> UnopenedCount, TotalMinorCompactions, HoldTime, AverageFilesPerTablet,
> Name]]
>
> At the ganglia Web UI, I can not see any Accumulo metrices for this
> cluster.I have checked every configurations hundreds of times.
> If anyone want, I can upload configurations, procedure or any required
> info. Please comment on what could be the problem in query execution?
> and how can I fix it?
> Is it a compatibility issue with Accumulo 1.7.0?
>
> Thanks
> Mohit Kaushik
>
>

Mime
View raw message