Mailing-List: contact chukwa-user-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: chukwa-user@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of zhu121972@163.com designates
 123.125.50.134 as permitted sender)
From: "ZJL" <zhu121972@163.com>
To: <chukwa-user@incubator.apache.org>
References: <2f29829c.78b.12c0efeb925.Coremail.zhu121972@163.com>
	<AANLkTik567mcKfuYn+JaMwCz9EEcZtKzoAX-jGKjn5H1@mail.gmail.com>
	<000001cb7cdc$13af8370$3b0e8a50$@com>
 <AANLkTikGzdhhomCV+UuddSf+49dh7-yJ82CbeF7hOmsK@mail.gmail.com>
In-Reply-To: <AANLkTikGzdhhomCV+UuddSf+49dh7-yJ82CbeF7hOmsK@mail.gmail.com>
Subject: RE: Data process for HICC
Date: Sun, 7 Nov 2010 19:19:37 +0800
Message-ID: <000001cb7e6d$aa13f200$fe3bd600$@com>
MIME-Version: 1.0
Content-Type: text/plain;
	charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Thread-Index: Act9P9pbTOs9GZZVToaveCwN4u/0gABLWcxg
Content-Language: zh-cn

HI eric :
   in my ubuntu ,I just install sysstat to collect system metrics.Your =
mean that if my ubuntu need to install sigar.=20
-----Original Message-----
From: chukwa-user-return-579-zhu121972=3D163.com@incubator.apache.org =
[mailto:chukwa-user-return-579-zhu121972=3D163.com@incubator.apache.org] =
On Behalf Of Eric Yang
Sent: 2010=E5=B9=B411=E6=9C=886=E6=97=A5 7:19
To: chukwa-user@incubator.apache.org
Subject: Re: Data process for HICC

1. For system metrics, it is likely the output of sar and iostat do
not match of what Chukwa expects.  I found system utilities output to
be highly unreliable for scrapping.  Hence, in Chukwa trunk, I have
moved to Sigar for collecting system metrics.  This should improve the
problem that you were seeing.  Your original question is about node
activity, and HDFS heatmap.  Those metrics are not populated
automatically.  For node activity, Chukwa was based on Torque's
pbsnodes.  This is no longer a maintained path.  For HDFS heatmap, you
need to have hdfs client trace and mr client trace log files stream
through Chukwa in order to generate graph for those metrics.  There is
no aggregation script to down sample the data for hdfs heatmap,
therefore only the last 6 hours is visible, if client trace log files
are processed by Chukwa.  There is a lot of work to change aggregation
from SQL to Pig+HBase.  However, most of the work is waiting for Pig
0.8 to be release in order for Chukwa to start the implementation.
Therefore, you might need to wait for a while for the features to
appear.

2.  hourlyRolling and dailyRolling should run automatically after
starting with start-all.sh script.

regards,
Eric

On Fri, Nov 5, 2010 at 4:24 AM, ZJL <zhu121972@163.com> wrote:
> HI eric:
> 1.In background,I have started dbAdmin and the dbAdmin.sh was =
running,otherwise the dbbase would have nothing.in my database ,some of =
field record have no data. not all. "System metrics collection may fail =
or be incomplete if your versions of sar and iostat do not match the =
ones that Chukwa expects" this citation come
> from chukwa releasenotes, i suspect if my sysstat version of ubuntu is =
not match for chukwa, if so, what can i do for that.
> 2.i don't know if hourlyRolling or dailyRolling automatically =
run,after starting bin/start-all.sh
>
> -----Original Message-----
> From: chukwa-user-return-576-zhu121972=3D163.com@incubator.apache.org =
[mailto:chukwa-user-return-576-zhu121972=3D163.com@incubator.apache.org] =
On Behalf Of Eric Yang
> Sent: 2010=E5=B9=B411=E6=9C=885=E6=97=A5 8:39
> To: chukwa-user@incubator.apache.org
> Subject: Re: Data process for HICC
>
> Hi,
>
> This may be caused by dbAdmin.sh was not running in the background.
> In Chukwa 0.4, you need to have dbAdmin.sh periodically create table
> partitions from the template tables.  If the script is not running,
> the data might not get loaded.
>
> I am not sure about your question about hourlyRolling or dailyRolling.
>  Those processes should be handled by data processor (./bin/chukwa
> dp).
>
> regards,
> Eric
>
> 2010/11/2 =E8=89=AF=E4=BA=BA <zhu121972@163.com>:
>>
>>  HI :    I always would like to use  chukwa to analyze the hadoop of
>> efficiency,but I  ran into several problems.
>>     firstly,i set up chukwa strictly following the instruction .my =
hicc work
>> normally and can display graph if there are some data in mysql for =
instance:
>> DFS Throughput Metrics,DFS Data Node Metrics,Cluster Metrics by =
Percentage.
>>     but some field record in mysql were not in mysql and they can not
>> display in hicc, for example: DFS Name Node Metrics,DFS FS ,Name =
System
>> Metrics,Map/Reduce Metircs,HDFS Heathmap(),Hadoop Activity,Event =
viewer,
>> Node Activity Graph.
>>   my configure:
>>   chukwa-hadoop-0.4.0-client.jar  have been in the hadoop=E2=80=99s =
lib
>>   both hadoop-metrics.properties and hadoop log4j.properties were in =
hadoop
>> of conf,i have list these documents in attachment.
>>   "System metrics collection may fail or be incomplete if your =
versions of
>> sar and iostat do not match the ones that Chukwa expects" this =
citation come
>> from chukwa releasenotes, i suspect  if my sysstat version of ubuntu =
is not
>> match for chukwa, if so, what can i do for that.
>>   could anybody give me some suggestions, thank you very much,
>>   by the way do anybody know how to start hourlyRolling and =
dailyRoilling in
>> 0.4.0 version and "Error initializing ChukwaClient with list of =
currently
>> registered adaptors, clearing our local list of adaptors" in logs,how =
can i
>> resolved it .
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
> __________ Information from ESET NOD32 Antivirus, version of virus =
signature database 5592 (20101104) __________
>
> The message was checked by ESET NOD32 Antivirus.
>
> http://www.eset.com
>
>
>
>
>

__________ Information from ESET NOD32 Antivirus, version of virus =
signature database 5592 (20101104) __________

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com