chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From NehaS Singh <>
Subject RE: Writing logs collected by chukwa to hbase
Date Mon, 07 Jan 2013 06:18:21 GMT

Whenever I run the command  $CHUKWA_HOME/bin/chukwa dumpArchive 'datatype=.*' 'hdfs://host:8020/chukwa/archive/*.arc'
the following output is displayed
"total of 0 files to search"

Neha Singh

From: Eric Yang []
Sent: Monday, January 07, 2013 2:48 AM
Subject: Re: Writing logs collected by chukwa to hbase

There are a few things that you need to do:

1. Configure the collector to use HBaseWriter.

Make sure that hadoop and hbase configuration are in the classpath of the collector.

2. Write a demux parser for your log file to store on HBase, and this runs in chukwa collector.

3. What log4j adaptors and appenders are you using to write the log files?
  I would recommend to use socket appender from Log4j and use socket adaptor to listen for
the incoming data stream.  This will ensure the records are delimited properly.

Hope this helps.


On Sun, Jan 6, 2013 at 6:00 AM, NehaS Singh <<>>
I am using chukwa 0.5. My probleam is want to store the logs from log4j or other party which
is not hadoop.Also i want to  write the logs to hbase.
But i am not ale to do so. I have configuerd file so i am able to see th
logs in hdfs.
what are the configuration or steps using which i can write the logs to hbase.
Please help.
From: Eric Yang [<>]
Sent: Saturday, January 05, 2013 11:55 PM
Subject: Re: Writing logs collected by chukwa to hbase

What version of Chukwa are you running?

On Fri, Dec 28, 2012 at 2:56 AM, NehaS Singh <<><<>>>
                I am using Chukwa to collect log4j logs which I can see in HDFS. But I am
not able to write it to HBASE. Can anyone help ?

Neha Singh

The contents of this e-mail and any attachment(s) may contain confidential or privileged information
for the intended recipient(s). Unintended recipients are prohibited from taking action on
the basis of information in this e-mail and using or disseminating the information, and must
notify the sender and delete it from their system. L&T Infotech will not accept responsibility
or liability for the accuracy or completeness of, or the presence of any virus or disabling
code in this e-mail"

View raw message