incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <eric...@gmail.com>
Subject Re: Unable to use Swimlanes visualization.
Date Wed, 03 Oct 2012 15:01:36 GMT
Hi Jesse,

Swimlanes and HDFS heatmap are disabled in Chukwa 0.5.  Those features were made with mySQL
queries.  In Chukwa 0.4, data were loaded to mySQL and visualization query mySQL database
to generate graphs.  In Chukwa 0.5, data storage was converted to HBase but not all data made
the transition.  There is still a lot of work to convert job data and HDFS client trace to
HBase, and update swimlanes and heatmap visualization code to read from HBase.  If anyone
would like to update those features to work with HBase, this is most welcome.

Sent from my iPhone

On Oct 3, 2012, at 1:24 AM, Jesse Chang <jesse.chang.2@gmail.com> wrote:

> Hi,
> 
> I am using Chukwa version 0.5.0 and the CDH4 package - "Version: 4.0.4 (#79 built by
jenkins on 20120821-1039 git: 119fe7452f0c356dcde763a1d22eda9363d24ad4)". I re-built  Chukwa
0.5.0 with cdh4 jar files - hbase-0.92.1-cdh4.0.0.jar, hadoop-common-2.0.0-cdh4.0.1.jar, hadoop-core-2.0.0-mr1-cdh4.0.1.jar,
and hadoop-hdfs-2.0.0-cdh4.0.1.jar.
> 
> I could see some data such as shuffle_output_bytes was collected at http://myhost:4080/hicc/jsp/graph_explorer.jsp.
I ran the aggregation script - share/chukwa/script/pig/ClusterSummary.pig - successfully to
see Cluster Status graphs and MapReduce graphs at http://myhost:4080/hicc/#. There are no
HDFS metrics graphs at the HDFS tab though.
> 
> First of all, is the Swimlanes visualization available at Chukwa 0.5.0? If not, what
is the current status?
> 
> How can I check if Demux is happening? I can see org.apache.hadoop.chukwa.extraction.demux.DemuxManager
is running, but I don't see anything is really happening. What would be the cause? How can
I debug it? Is Demux still necessary to see a swimlanes visualization of Mapreduce stages?
> 
> Basically I followed the guide at http://incubator.apache.org/chukwa/docs/r0.5.0/Quick_Start_Guide.html
and http://incubator.apache.org/chukwa/docs/r0.5.0/admin.html. I just found that Chukwa Demux
was removed from the picture when I compared it to the 0.4.0 guide at http://incubator.apache.org/chukwa/docs/r0.4.0/admin.html.
Still does Chukwa 0.5.0 support Demux?
> 
> What would be the best way to see swimlanes of mapreduce stages?
> 
> Thanks,
> Jesse
> 
> 
> 
> 
> 

Mime
View raw message