incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <eric...@gmail.com>
Subject Re: Df Demux processor not able to write to hbase
Date Fri, 29 Jul 2011 05:36:55 GMT
In your application, you will need to setup logging through
ChukwaDailyRollingFileAppender to ensure the log entries are delimited
by control-A character for a full log entry or use Log4JSocketAppender
for streaming directly to SocketAdaptor.  Otherwise,
CharFileTailingAdaptorUTF8NewLineEscaped adaptor may mistake
multi-line output as beginning of a log entry.
Hope this helps.

regards,
Eric

On Thu, Jul 28, 2011 at 2:02 PM, AD <abhijit.dhar.in@gmail.com> wrote:
> Thanks for the reply. I was basically just playing around with setting up
> hbase. But, I got the error sorted out now.
>
> I was wondering if you could tell me what is the best adapter and demux
> processor combination to use to read log file and put it into hbase?
> Here is a sample:
> 2011-07-28 00:55:26,129 INFO [main]
> [fetch.mapreduce.FetchMapper.logHeapUsage()] - Fetcher feeding queue
> manager. Heap usage: 160474424 out of 778502144 bytes.
> 2011-07-28 00:55:27,133 INFO [main] [fetch.mapreduce.FetchMapper.doWork()] -
> -activeThreads=1, spinWaiting=0, fetchQueues.totalSize=0
> 2011-07-28 00:55:27,134 INFO [main]
> [fetch.mapreduce.FetchMapper.feedQueueManager()] - feeding 0 input urls ...
>
> I'm trying to use 'TsProcessor' and
> 'filetailer.CharFileTailingAdaptorUTF8NewLineEscaped', But I get :
>
> java.text.ParseException: Unparseable date
>
> It works fine though with filetailer.LWFTAdaptor, but it takes whole file as
> chunk and doesn't parse line by line.
>
> Thanks much.
>
> --
> View this message in context: http://apache-chukwa.679492.n3.nabble.com/Df-Demux-processor-not-able-to-write-to-hbase-tp3205942p3207841.html
> Sent from the Chukwa - Users mailing list archive at Nabble.com.
>

Mime
View raw message