chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shouvanik Haldar <shouvanik.hal...@gmail.com>
Subject Re: Writing logs collected by chukwa to hbase
Date Tue, 08 Jan 2013 13:04:10 GMT
I am not able to see any thing on hdfs:
when I browse using the url ->
http://ec2-54-234-57-44.compute-1.amazonaws.com:50070/nn_browsedfscontent.jsp
I see nothing..but

" Network Error (tcp_error)

  A communication error occurred: "Operation timed out"   The Web Server
may be down, too busy, or experiencing other problems preventing it from
responding to requests. You may wish to try again at a later time.
For assistance, contact your network support team. "




On Tue, Jan 8, 2013 at 6:22 PM, Shouvanik Haldar <shouvanik.haldar@gmail.com
> wrote:

> How I was sure data getting collected at collector, is by seeing the below
> statistics
>
>
> Date:1357649516006
> Now:1357649518803
> numberHTTPConnection in time window:2
> numberchunks in time window:7
> lifetimechunks:2612
>
>
>
>
> On Tue, Jan 8, 2013 at 6:20 PM, Shouvanik Haldar <
> shouvanik.haldar@gmail.com> wrote:
>
>> Hi,
>>
>> I was following the post -> http://www.undercloud.org/?p=429. I did the
>> steps as was mentioned here. But, when I was going to check in HDFS whether
>> data has come from chukwa, I cannot see the page. I get the exception as
>>
>>
>>
>> Also, I want to know what is the default location of chukwa logs. I found
>> out a location ->
>> "/home/ec2-user/chukwa/chukwa-incubating-0.5.0/sbin/logs". Is this the
>> location? Previously, I had some logs inside this directory. I have deleted
>> these logs. Now, no logs are getting created.
>>
>> Previously, when logs were existing, I checked one log entry. It says
>> "nice: /home/ec2-user/chukwa/chukwa-incubating-0.5.0/sbin/bin/chukwa: No
>> such file or directory".
>>
>>
>> I have edited bash_profile to give the path of CHUKWA_HOME. Is this
>> required?
>>
>> I am using "chukwa-incubating-0.5.0" and "hadoop-1.0.4". I can see data
>> getting collected at the collector by browsing the url =>
>> http://ec2-54-234-57-44.compute-1.amazonaws.com:8080/chukwa?ping=true
>>
>>
>>
>>
>>
>> But somehow, data not getting inside hdfs. Please help.
>>
>>
>> Regards,
>> Shouvanik
>>
>>
>> On Tue, Jan 8, 2013 at 10:22 AM, Eric Yang <eric818@gmail.com> wrote:
>>
>>> The quick start guide may help:
>>>
>>> http://incubator.apache.org/chukwa/docs/r0.5.0/Quick_Start_Guide.html
>>>
>>> regards,
>>> Eric
>>>
>>>
>>> On Mon, Jan 7, 2013 at 1:17 AM, Shouvanik Haldar <
>>> shouvanik.haldar@gmail.com> wrote:
>>>
>>>> Hi,
>>>> I want to install chukwa on a single node and test it. Where can I find
>>>> a suitable document?
>>>>
>>>>
>>>> On Mon, Jan 7, 2013 at 2:47 AM, Eric Yang <eric818@gmail.com> wrote:
>>>>
>>>>> There are a few things that you need to do:
>>>>>
>>>>> 1. Configure the collector to use HBaseWriter.
>>>>>
>>>>>   http://incubator.apache.org/chukwa/docs/r0.5.0/collector.html
>>>>>
>>>>> Make sure that hadoop and hbase configuration are in the classpath of
>>>>> the collector.
>>>>>
>>>>> 2. Write a demux parser for your log file to store on HBase, and this
>>>>> runs in chukwa collector.
>>>>>
>>>>>   http://incubator.apache.org/chukwa/docs/r0.5.0/programming.html
>>>>>
>>>>> 3. What log4j adaptors and appenders are you using to write the log
>>>>> files?
>>>>>   I would recommend to use socket appender from Log4j and use socket
>>>>> adaptor to listen for the incoming data stream.  This will ensure the
>>>>> records are delimited properly.
>>>>>
>>>>> Hope this helps.
>>>>>
>>>>> regards,
>>>>> Eric
>>>>>
>>>>> On Sun, Jan 6, 2013 at 6:00 AM, NehaS Singh <
>>>>> NehaS.Singh@lntinfotech.com> wrote:
>>>>>
>>>>>> I am using chukwa 0.5. My probleam is want to store the logs from
>>>>>> log4j or other party which is not hadoop.Also i want to  write the
logs to
>>>>>> hbase.
>>>>>> But i am not ale to do so. I have configuerd log4j.properties file
so
>>>>>> i am able to see th logs in hdfs.
>>>>>> what are the configuration or steps using which i can write the logs
>>>>>> to hbase.
>>>>>> Please help.
>>>>>> ________________________________________
>>>>>> From: Eric Yang [eric818@gmail.com]
>>>>>> Sent: Saturday, January 05, 2013 11:55 PM
>>>>>> To: chukwa-user@incubator.apache.org
>>>>>> Subject: Re: Writing logs collected by chukwa to hbase
>>>>>>
>>>>>> What version of Chukwa are you running?
>>>>>>
>>>>>> regards,
>>>>>> Eric
>>>>>>
>>>>>> On Fri, Dec 28, 2012 at 2:56 AM, NehaS Singh <
>>>>>> NehaS.Singh@lntinfotech.com<mailto:NehaS.Singh@lntinfotech.com>>
>>>>>> wrote:
>>>>>> Hi,
>>>>>>                 I am using Chukwa to collect log4j logs which I can
>>>>>> see in HDFS. But I am not able to write it to HBASE. Can anyone help
?
>>>>>>
>>>>>> Regards,
>>>>>> Neha Singh
>>>>>>
>>>>>>
>>>>>> ________________________________
>>>>>> The contents of this e-mail and any attachment(s) may contain
>>>>>> confidential or privileged information for the intended recipient(s).
>>>>>> Unintended recipients are prohibited from taking action on the basis
of
>>>>>> information in this e-mail and using or disseminating the information,
and
>>>>>> must notify the sender and delete it from their system. L&T Infotech
will
>>>>>> not accept responsibility or liability for the accuracy or completeness
of,
>>>>>> or the presence of any virus or disabling code in this e-mail"
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Thanks,
>>>> *Shouvanik*
>>>>
>>>
>>>
>>
>>
>> --
>> Thanks,
>> *Shouvanik*
>>
>
>
>
> --
> Thanks,
> *Shouvanik*
>



-- 
Thanks,
*Shouvanik*

Mime
View raw message