incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 孙长 <smart02_...@163.com>
Subject Re:Re: Re: Re: hadoop metrics isn't collected into mysql
Date Sat, 02 Jul 2011 04:16:31 GMT
yes, there are rpc and mapred logs in  log file
hadoop-metrics.properties is like this, and hadoop version is 0.2
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.


chukwaAgent.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext
chukwaAgent.period=60
chukwaAgent.directory=/tmp/chukwa/log/metrics/
 
chukwaHttpSender.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext
chukwaHttpSender.period=60
chukwaHttpSender.directory=/tmp/chukwa/log/metrics/


chunkQueue.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext
chunkQueue.period=60
chunkQueue.directory=/tmp/chukwa/log/metrics/


dfs.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext
dfs.period=60
dfs.directory=/tmp/chukwa/log/metrics/
dfs.uuid=true


jvm.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext
jvm.period=60
jvm.directory=/tmp/chukwa/log/metrics/
jvm.uuid=true


mapred.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext
mapred.period=60
mapred.directory=/tmp/chukwa/log/metrics/
mapred.uuid=true


rpc.class=org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext
rpc.period=60
rpc.directory=/tmp/chukwa/log/metrics/
rpc.uuid=true



At 2011-07-02 11:39:32,"Eric Yang" <eric818@gmail.com> wrote:

>Check the log directory and see if you see any log file with rpc or
>mapred in the file name?
>What does the hadoop-metrics.properties look like?  What version of
>hadoop are you running?
>
>regards,
>Eric
>
>2011/7/1 孙长 <smart02_ren@163.com>:
>> i know where the problem is , i didn't put log4j and hadoop-metrics file
>> into all  nodes' hadoop conf dir.
>> Now the problem disapears
>> but for hadoop metrics, i only got dfs data in mysql, no mapreduce, jvm, rpc
>> data, why?
>> thank you~
>>
>>
>> At 2011-07-01 22:59:41,"Eric Yang" <eric818@gmail.com> wrote:
>>
>>>Looks like mapreduce can not find json.jar in its class path.  Which
>>>version of Hadoop are you using?
>>>
>>>regards,
>>>Eric
>>>
>>>2011/7/1 孙长 <smart02_ren@163.com>:
>>>> hi,i have copied chukwa-hadoop-*-client.jar and json.jar to
>>>> HADOOP_HOME/lib, and restarted hadoop, but the MR job lanched by demux runs
>>>> error:
>>>> task error is as follows:
>>>>
>>>> Error: java.lang.ClassNotFoundException: org.json.JSONException
>>>> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>>>> 	at java.security.AccessController.doPrivileged(Native Method)
>>>> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>>>> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>>> 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:252)
>>>> 	at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:320)
>>>> 	at java.lang.Class.forName0(Native Method)
>>>> 	at java.lang.Class.forName(Class.java:169)
>>>> 	at
>>>> org.apache.hadoop.chukwa.extraction.demux.processor.mapper.MapProcessorFactory.getProcessor(MapProcessorFactory.java:40)
>>>> 	at
>>>> org.apache.hadoop.chukwa.extraction.demux.Demux$MapClass.map(Demux.java:95)
>>>> 	at
>>>> org.apache.hadoop.chukwa.extraction.demux.Demux$MapClass.map(Demux.java:64)
>>>> 	at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
>>>> 	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
>>>> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
>>>> 	at org.apache.hadoop.mapred.Child.main(Child.java:170)
>>>>
>>>> At 2011-07-01 13:17:42,"Eric Yang" <eric818@gmail.com> wrote:
>>>>
>>>>>Did you copy chukwa-hadoop-*-client.jar and json.jar to
>>>>>HADOOP_HOME/lib, and restart hadoop?
>>>>>
>>>>>regards,
>>>>>Eric
>>>>>
>>>>>2011/6/30 孙长 <smart02_ren@163.com>:
>>>>>> Hi, I am using chukwa 0.4 , and i start a collector and a agent on
two
>>>>>> nodes, but i only got system metrics in mysql, no hadoop metrics
like jvm,
>>>>>> dfs, mapred
>>>>>> dbAdmin.sh is running background, log4j.properties
>>>>>> and hadoop-metrics.properties in hadoop conf dir are set according
to user
>>>>>> guide
>>>>>> no dfs, mapred logs in chukwa/log/metrics dir.
>>>>>> thank you ~
>>>>>>
>>>>>>
>>>>
>>>>
>>>>
>>
>>
>>
Mime
View raw message