incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Yang <ey...@yahoo-inc.com>
Subject Re: How to obtain the chunk time stamp while processing the chunk using a Socket Program
Date Fri, 03 Jun 2011 17:42:41 GMT
In Chukwa Chunk data structure, we have a meta data field called tags, which allows Chukwa
agent/adaptor to inject data into this field.

We currently put cluster="" into the tag field in Chukwa Agent.  It is possible to add a tag
for Chukwa adaptor to send over the timestamp.

Regards,
Eric

On 6/3/11 5:21 AM, "DKN" <devaprasad@in.ibm.com> wrote:

I implemented a simple Socket program to listen to the collector on 9094
port, so that I can process that information further (using the
SocketTeeWriter).

I used this type of a command to tap into my specific data type :

HEADER datatype=SysOut*\n

I was able to get the chunk as RAW data. But it didn't have any information
about the time stamp which would be used while persisting in HDFS/HBase. Is
it possible to send the time stamp information of the chunk in the header ?
{ I am basically wondering about data consistency if I persist this
information else where apart from HDFS .. }

Cheers, DKN

--
View this message in context: http://apache-chukwa.679492.n3.nabble.com/How-to-obtain-the-chunk-time-stamp-while-processing-the-chunk-using-a-Socket-Program-tp3019416p3019416.html
Sent from the Chukwa - Users mailing list archive at Nabble.com.


Mime
View raw message