incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Graham <billgra...@gmail.com>
Subject Re: How to use metadata sent by agent to collector
Date Fri, 01 Oct 2010 17:04:25 GMT
capp is chunk.streamName
csource is chunk.source
ctags is chunk.tags

I believe body is chunk.data but note that a chunk can hold data from
multiple records so you'd need to check chunk.recordOffsets to know
how to split it up into multiple records.


On Fri, Oct 1, 2010 at 9:32 AM, Stuti Awasthi
<Stuti_Awasthi@persistent.co.in> wrote:
> Hi Bill,
>
> I wanted to access the "maps : " fileds
> "mapFields": {
>
> "capp": "/home/hadoop/xend.log",
> "csource": "iaas-1-0-16-3e-45-46-7d.iaas.torolab.ibm.com",
> "ctags": " cluster=\"chukwa",
>  "body" : "xyz"
> }
>
> I checked that ChunkImpl class sets all these values. I am looking into the correct code?
If yes, I can use relevant key value pair from here.
>
> Please Suggest
> Regards
> Stuti
>
> -----Original Message-----
> From: Bill Graham [mailto:billgraham@gmail.com]
> Sent: Friday, October 01, 2010 9:55 PM
> To: chukwa-user@incubator.apache.org
> Subject: Re: How to use metadata sent by agent to collector
>
> Can you be more specific regarding what you're trying to do? The
> ServlerCollector, Demux and Processor classes all have access to the
> Chunk object which encapsulated the fields you list, so it's not clear
> what field access you're missing.
>
> On Fri, Oct 1, 2010 at 6:15 AM, Stuti Awasthi
> <Stuti_Awasthi@persistent.co.in> wrote:
>> Hi all,
>>
>>
>>
>> I noticed that when agent sends chunks to collector it also sends the
>> metadata along with it, but in demux.java class , we are just dealing with
>> the record entry that comes from agent.
>>
>> How can I use this metadata information that is sent by agent to collector
>> also on collector side.
>>
>>
>>
>> Example , each chunk sends the meta data given as below , how can I use
>> other fields like Key  or capp on my collector side.
>>
>> {
>>
>>     "DataType": "logtype",
>>
>>     "Key":
>> "1285916400000/iaas-1-0-16-3e-45-46-7d.iaas.torolab.ibm.com/1285917710741",
>>
>>     "Timestamp": 1285917710741,
>>
>>     "mapFields": {
>>
>>   "capp": "/home/hadoop/xend.log",
>>
>>   "csource": "iaas-1-0-16-3e-45-46-7d.iaas.torolab.ibm.com",
>>
>>   "ctags": " cluster=\"chukwa",
>>
>>  "body" : "xyz"
>>
>> }
>>
>>
>>
>> Any pointers will be helpful.
>>
>>
>>
>> Regards
>>
>> Stuti Awasthi
>>
>> DISCLAIMER ========== This e-mail may contain privileged and confidential
>> information which is the property of Persistent Systems Ltd. It is intended
>> only for the use of the individual or entity to which it is addressed. If
>> you are not the intended recipient, you are not authorized to read, retain,
>> copy, print, distribute or use this message. If you have received this
>> communication in error, please notify the sender and delete all copies of
>> this message. Persistent Systems Ltd. does not accept any liability for
>> virus infected mails.
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is the property
of Persistent Systems Ltd. It is intended only for the use of the individual or entity to
which it is addressed. If you are not the intended recipient, you are not authorized to read,
retain, copy, print, distribute or use this message. If you have received this communication
in error, please notify the sender and delete all copies of this message. Persistent Systems
Ltd. does not accept any liability for virus infected mails.
>

Mime
View raw message