chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <>
Subject Re: How to use TsProcessor ?
Date Mon, 13 Jun 2011 16:53:30 GMT
Hi Bill,

Must be early Monday morning.  I was reading the javadoc,but misread it.  Please discard my
The Schema should be:

TableName: TsProcessor
ColumnFamily: log


On 6/13/11 9:47 AM, "Bill Graham" <> wrote:

> TsProcessor.time.regex.[some_data_type] maps to ColumnFamily

Eric, you lost me here. Why does the regular expression map to a column family?

On Mon, Jun 13, 2011 at 9:38 AM, Eric Yang <> wrote:
Hbase schema is defined by annotation in demux  parsers.  For TsProcessor, it is a generic
parser, it is not targeting a specific data type.  HBaseWriter is currently not handling this
generic parser well.
The current implementation is writing data processed by TsProcessor to:

TableName: TsProcessor
ColumnFamily: log

It may be possible to add handling of TsProcessor with this specification:

Chunk DataType maps to TableName
TsProcessor.time.regex.[some_data_type] maps to ColumnFamily

If this is reasonable implementation, please file a jira.  Thanks


On 6/13/11 8:56 AM, "Bill Graham" < <>
> wrote:

Apologies, but the documentation around processor configs is somewhat out dated (CHUKWA-538).

For one, TsProcessor is not the default. DefaultProcessor is. You can change this with the
chukwa.demux.mapper.default.processor setting.

Also, ithe docs don't include some enhancements to TsProcessor to make it more multi-purpose.
It can handle any data type passed to it as long as it can parse out the date. See this JIRA
for how to override the default or the per-dataType date parsing logic:

Eric is best to field the HBase schema question.

On Mon, Jun 13, 2011 at 1:57 AM, DKN < <>
> wrote:
I am just wondering if TsProcessor is a generic de-mux processor ? Is there a
documentation for what is TsProcessor and if there is a customizable way of
using it for generic data types defined ? I wanted to extend a demux
processor for log processing and came to know that TsProcessor is default in
this wiki link :

I also couldn't see this table definition for HBase (in the hbase.schema).
Where can I find the schema that I can create the table definitions in HBase

Thanks in advance.

Cheers, DKN

View this message in context:
Sent from the Chukwa - Users mailing list archive at

View raw message