Return-Path: X-Original-To: apmail-incubator-chukwa-user-archive@www.apache.org Delivered-To: apmail-incubator-chukwa-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C077CD923 for ; Tue, 8 Jan 2013 04:52:42 +0000 (UTC) Received: (qmail 49787 invoked by uid 500); 8 Jan 2013 04:52:42 -0000 Delivered-To: apmail-incubator-chukwa-user-archive@incubator.apache.org Received: (qmail 49758 invoked by uid 500); 8 Jan 2013 04:52:42 -0000 Mailing-List: contact chukwa-user-help@incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: chukwa-user@incubator.apache.org Delivered-To: mailing list chukwa-user@incubator.apache.org Received: (qmail 49744 invoked by uid 99); 8 Jan 2013 04:52:42 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Jan 2013 04:52:42 +0000 X-ASF-Spam-Status: No, hits=2.7 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_REPLY,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of eric818@gmail.com designates 209.85.214.181 as permitted sender) Received: from [209.85.214.181] (HELO mail-ob0-f181.google.com) (209.85.214.181) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 Jan 2013 04:52:37 +0000 Received: by mail-ob0-f181.google.com with SMTP id oi10so17985408obb.40 for ; Mon, 07 Jan 2013 20:52:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=aQxLZu9wRN6uXEKwj7hA1/oC5XyOjVGJ3N1NLiONa5o=; b=LCAIa+Dby/XbldtmAxlkzp+DCDTCPei0+iJladBdN2Ne3yhK3M53FjP3k9crlkJU85 L287goVmszPrdQom6lXjba9URcd/nRamF62KsWrUZW9xdW+gmnB0z2Db8AfvMzCDHfnl jjME6XdOOPT+UxnRmtOzluMEtzmIlojuEGR55BXEUmNAnLNgX8RFdh8sMhRCrd57BFbF 1uU0IU6Oc1sjOn7zGQT5wzJvfBylTuo9jpuExdm3lppZ4qbKPBfABL9GhCBedup+5GGi wVkuTt4ATXFyw9G2c5U39wNacAklFfsLdprhmyB0QqFHu6TtNnsu8U4LXqLjuidT2hCk nmhQ== MIME-Version: 1.0 Received: by 10.60.31.205 with SMTP id c13mr34219335oei.135.1357620736618; Mon, 07 Jan 2013 20:52:16 -0800 (PST) Received: by 10.76.34.105 with HTTP; Mon, 7 Jan 2013 20:52:16 -0800 (PST) In-Reply-To: References: Date: Mon, 7 Jan 2013 20:52:16 -0800 Message-ID: Subject: Re: Writing logs collected by chukwa to hbase From: Eric Yang To: chukwa-user@incubator.apache.org Content-Type: multipart/alternative; boundary=e89a8f839dbfbef19e04d2bfb55c X-Virus-Checked: Checked by ClamAV on apache.org --e89a8f839dbfbef19e04d2bfb55c Content-Type: text/plain; charset=ISO-8859-1 The quick start guide may help: http://incubator.apache.org/chukwa/docs/r0.5.0/Quick_Start_Guide.html regards, Eric On Mon, Jan 7, 2013 at 1:17 AM, Shouvanik Haldar wrote: > Hi, > I want to install chukwa on a single node and test it. Where can I find a > suitable document? > > > On Mon, Jan 7, 2013 at 2:47 AM, Eric Yang wrote: > >> There are a few things that you need to do: >> >> 1. Configure the collector to use HBaseWriter. >> >> http://incubator.apache.org/chukwa/docs/r0.5.0/collector.html >> >> Make sure that hadoop and hbase configuration are in the classpath of the >> collector. >> >> 2. Write a demux parser for your log file to store on HBase, and this >> runs in chukwa collector. >> >> http://incubator.apache.org/chukwa/docs/r0.5.0/programming.html >> >> 3. What log4j adaptors and appenders are you using to write the log files? >> I would recommend to use socket appender from Log4j and use socket >> adaptor to listen for the incoming data stream. This will ensure the >> records are delimited properly. >> >> Hope this helps. >> >> regards, >> Eric >> >> On Sun, Jan 6, 2013 at 6:00 AM, NehaS Singh wrote: >> >>> I am using chukwa 0.5. My probleam is want to store the logs from log4j >>> or other party which is not hadoop.Also i want to write the logs to hbase. >>> But i am not ale to do so. I have configuerd log4j.properties file so i >>> am able to see th logs in hdfs. >>> what are the configuration or steps using which i can write the logs to >>> hbase. >>> Please help. >>> ________________________________________ >>> From: Eric Yang [eric818@gmail.com] >>> Sent: Saturday, January 05, 2013 11:55 PM >>> To: chukwa-user@incubator.apache.org >>> Subject: Re: Writing logs collected by chukwa to hbase >>> >>> What version of Chukwa are you running? >>> >>> regards, >>> Eric >>> >>> On Fri, Dec 28, 2012 at 2:56 AM, NehaS Singh < >>> NehaS.Singh@lntinfotech.com> wrote: >>> Hi, >>> I am using Chukwa to collect log4j logs which I can see >>> in HDFS. But I am not able to write it to HBASE. Can anyone help ? >>> >>> Regards, >>> Neha Singh >>> >>> >>> ________________________________ >>> The contents of this e-mail and any attachment(s) may contain >>> confidential or privileged information for the intended recipient(s). >>> Unintended recipients are prohibited from taking action on the basis of >>> information in this e-mail and using or disseminating the information, and >>> must notify the sender and delete it from their system. L&T Infotech will >>> not accept responsibility or liability for the accuracy or completeness of, >>> or the presence of any virus or disabling code in this e-mail" >>> >>> >> > > > -- > Thanks, > *Shouvanik* > --e89a8f839dbfbef19e04d2bfb55c Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable The quick start guide may help:

regards,
Eric

On Mon= , Jan 7, 2013 at 1:17 AM, Shouvanik Haldar <shouvanik.haldar@gmai= l.com> wrote:
Hi,
I want to= install chukwa on a single node and test it. Where can I find a suitable d= ocument?


On Mon, Jan 7, 2013 at 2:47 AM, Eric Yang <eric818@gmail.= com> wrote:
There are a few things that you need to= do:

1. Configure the collector to use HBaseWriter= .


Make sure that hadoop and hbase configuration are in th= e classpath of the collector.

2. Write a demux par= ser for your log file to store on HBase, and this runs in chukwa collector.=


3. What log4j adaptors an= d appenders are you using to write the log files?
=A0 I would recommend to use socket appender from Log4j and use socket= adaptor to listen for the incoming data stream. =A0This will ensure the re= cords are delimited properly.

Hope this helps.

regards,
Eric

On Sun, Jan 6, 2013 at 6:00 AM, NehaS Singh <NehaS.Singh@lntinfotech.com> wrote:
I am using chukwa 0.5. My probleam is want t= o store the logs from log4j or other party which is not hadoop.Also i want = to =A0write the logs to hbase.
But i am not ale to do so. I have configuerd log4j.properties file so i am = able to see th logs in hdfs.
what are the configuration or steps using which i can write the logs to hba= se.
Please help.
________________________________________
From: Eric Yang [eri= c818@gmail.com]
Sent: Saturday, January 05, 2013 11:55 PM
To: c= hukwa-user@incubator.apache.org
Subject: Re: Writing logs collected by chukwa to hbase

What version of Chukwa are you running?

regards,
Eric

On Fri, Dec 28, 2012 at 2:56 AM, NehaS Singh <NehaS.Singh@lntinfot= ech.com<mailto:NehaS.Singh@lntinfotech.com>> wrote:
Hi,
=A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 I am using Chukwa to collect log4j logs whi= ch I can see in HDFS. But I am not able to write it to HBASE. Can anyone he= lp ?

Regards,
Neha Singh


________________________________
The contents of this e-mail and any attachment(s) may contain confidential = or privileged information for the intended recipient(s). Unintended recipie= nts are prohibited from taking action on the basis of information in this e= -mail and using or disseminating the information, and must notify the sende= r and delete it from their system. L&T Infotech will not accept respons= ibility or liability for the accuracy or completeness of, or the presence o= f any virus or disabling code in this e-mail"





--
Thanks,
Shouvanik

--e89a8f839dbfbef19e04d2bfb55c--