incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From AD <straightfl...@gmail.com>
Subject Re: Chukwa trunk does not write to HBase
Date Wed, 26 Oct 2011 12:48:28 GMT
interesting.  So demux is now part of the collector and does not need to be
run as another job?

Since demux is doing basic ETL into hbase, is there a way to actually parse
the fields of the log record and insert those into sep fields into Hbase to
run mapreduce there (instead of the whole body as a field is what I think is
happening)

On Wed, Oct 26, 2011 at 12:59 AM, SALAMI Patrick <salami@systransoft.com>wrote:

> Ok, thanks!
>
> -----Original Message-----
> From: Eric Yang [mailto:eyang@hortonworks.com] On Behalf Of Eric Yang
> Sent: Tuesday, October 25, 2011 6:19 PM
> To: chukwa-user@incubator.apache.org
> Subject: Re: Chukwa trunk does not write to HBase
>
> Demuxer for Chukwa 0.4 is a mapreduce job acting as ETL process to convert
> the data into semi structure format for further processing.  For Chukwa
> trunk with HBase, demux is running as part of collector for ETL.  Hence,
> there is no need to run demux process.
>
> regards,
> Eric
>
> On Oct 25, 2011, at 3:49 PM, SALAMI Patrick wrote:
>
> > Also, while we are chatting, I was hoping to understand the role of the
> demuxer. I am assuming that HICC pulls all of its data from HBase. If using
> HBase, is it still necessary to run the demuxer? I didn't see any mention of
> it in the latest quick start guide.
> >
> > Thanks!
> >
> > Patrick
> >
> > -----Original Message-----
> > From: Eric Yang [mailto:eyang@hortonworks.com] On Behalf Of Eric Yang
> > Sent: Tuesday, October 25, 2011 2:45 PM
> > To: chukwa-user@incubator.apache.org
> > Subject: Re: Chukwa trunk does not write to HBase
> >
> > Hadoop trunk will require a different configuration than the one
> described in the Quick_Start guide.
> >
> > 1. Apply this patch: HADOOP-7436, and rebuild hadoop.
> > 2. Copy hadoop-metrics2.properties enclosed in this message to
> HADOOP_CONF_DIR
> >
> > Restart hadoop.  You might need to match the HBase table schema with the
> metrics emitted by Hadoop metrics 2 framework.
> > Hope this helps.
> >
> >
> > This message contains confidential information and is intended only for
> the individual(s) named. If you are not the named addressee you should not
> disseminate, distribute or copy this e-mail. Please notify the sender
> immediately by e-mail if you have received this e-mail by mistake and delete
> this e-mail from your system.
>
>
> This message contains confidential information and is intended only for the
> individual(s) named. If you are not the named addressee you should not
> disseminate, distribute or copy this e-mail. Please notify the sender
> immediately by e-mail if you have received this e-mail by mistake and delete
> this e-mail from your system.
>

Mime
View raw message