incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <ey...@yahoo-inc.com>
Subject Re: How to use TsProcessor ?
Date Mon, 13 Jun 2011 16:38:07 GMT
Hbase schema is defined by annotation in demux  parsers.  For TsProcessor, it is a generic
parser, it is not targeting a specific data type.  HBaseWriter is currently not handling this
generic parser well.
The current implementation is writing data processed by TsProcessor to:

TableName: TsProcessor
ColumnFamily: log

It may be possible to add handling of TsProcessor with this specification:

Chunk DataType maps to TableName
TsProcessor.time.regex.[some_data_type] maps to ColumnFamily

If this is reasonable implementation, please file a jira.  Thanks

Regards,
Eric

On 6/13/11 8:56 AM, "Bill Graham" <billgraham@gmail.com> wrote:

Apologies, but the documentation around processor configs is somewhat out dated (CHUKWA-538).

For one, TsProcessor is not the default. DefaultProcessor is. You can change this with the
chukwa.demux.mapper.default.processor setting.

https://issues.apache.org/jira/browse/CHUKWA-473

Also, ithe docs don't include some enhancements to TsProcessor to make it more multi-purpose.
It can handle any data type passed to it as long as it can parse out the date. See this JIRA
for how to override the default or the per-dataType date parsing logic:

https://issues.apache.org/jira/browse/CHUKWA-472


Eric is best to field the HBase schema question.


On Mon, Jun 13, 2011 at 1:57 AM, DKN <devaprasad@in.ibm.com> wrote:
I am just wondering if TsProcessor is a generic de-mux processor ? Is there a
documentation for what is TsProcessor and if there is a customizable way of
using it for generic data types defined ? I wanted to extend a demux
processor for log processing and came to know that TsProcessor is default in
this wiki link : http://wiki.apache.org/hadoop/DemuxModification

I also couldn't see this table definition for HBase (in the hbase.schema).
Where can I find the schema that I can create the table definitions in HBase
..

Thanks in advance.

Cheers, DKN

--
View this message in context: http://apache-chukwa.679492.n3.nabble.com/How-to-use-TsProcessor-tp3058006p3058006.html
Sent from the Chukwa - Users mailing list archive at Nabble.com.



Mime
View raw message