incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From AD <straightfl...@gmail.com>
Subject Re: Writing custom parser
Date Mon, 26 Dec 2011 17:54:29 GMT
Also I wrote my own processor by just mimicking one of the existing like
TSProcessor.  Its pretty simple you only need to implement a couple methods
in a new classfile.  Here is a massively oversimplified example.

package org.apache.hadoop.chukwa.extraction.demux.processor.mapper;

public class MyCustomProcessor extends AbstractProcessor {

@Override
  protected void parse(String recordEntry,OutputCollector<ChukwaRecordKey,
ChukwaRecord> output, Reporter reporter) {

  ChukwaRecord record = new ChukwaRecord();
        this.buildGenericRecord(record, recordEntry, archiveKey
            .getTimePartition(), chunk.getDataType());

   // Do all your parsing here then add whatever fields you want to the
record

   record.add("some_field","some_value");
   output.collect(key,record);
 }
}


On Mon, Dec 26, 2011 at 12:24 PM, Eric Yang <eric818@gmail.com> wrote:

> To store data to HBase, the demux parser must be implanted with Chukwa
> HBase annotation.
> For more details, check out this document:
>
> http://people.apache.org/~eyang/chukwa-0.5.0-docs/programming.html
>
> Section: Demux Data To HBase
>
> Hope this helps.
>
> regards,
> Eric
>
> On Mon, Dec 26, 2011 at 5:47 AM, Mohammad Tariq <dontariq@gmail.com>
> wrote:
> > Hello list,
> >
> >    I have used Chukwa, without any modification till now. Now I want
> > to collect data that is in form of "xml" and store it in Hbase tables
> > and for that I have to write the parser.Since it is my first attempt
> > of doing such thing, I would like to have some guidance from the
> > experts. I am using this link for getting started
> > "http://wiki.apache.org/hadoop/DemuxModification".If anyone is aware
> > of some other place that can help me, please let me know.Many thanks.
> >
> > Regards,
> >     Mohammad Tariq
>

Mime
View raw message