incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Graham <billgra...@gmail.com>
Subject Re: How to use metadata sent by agent to collector
Date Fri, 01 Oct 2010 18:26:58 GMT
No problem. Also note that you should be able to get the same fields
from the ChukwaRecord object if you have access to it, which would
probably be easier.

On Fri, Oct 1, 2010 at 10:52 AM, Stuti Awasthi
<Stuti_Awasthi@persistent.co.in> wrote:
> Thanks Bill, I will try using chunk object.
>
> -----Original Message-----
> From: Bill Graham [mailto:billgraham@gmail.com]
> Sent: Friday, October 01, 2010 10:34 PM
> To: Stuti Awasthi
> Cc: chukwa-user@incubator.apache.org
> Subject: Re: How to use metadata sent by agent to collector
>
> capp is chunk.streamName
> csource is chunk.source
> ctags is chunk.tags
>
> I believe body is chunk.data but note that a chunk can hold data from
> multiple records so you'd need to check chunk.recordOffsets to know
> how to split it up into multiple records.
>
>
> On Fri, Oct 1, 2010 at 9:32 AM, Stuti Awasthi
> <Stuti_Awasthi@persistent.co.in> wrote:
>> Hi Bill,
>>
>> I wanted to access the "maps : " fileds
>> "mapFields": {
>>
>> "capp": "/home/hadoop/xend.log",
>> "csource": "iaas-1-0-16-3e-45-46-7d.iaas.torolab.ibm.com",
>> "ctags": " cluster=\"chukwa",
>>  "body" : "xyz"
>> }
>>
>> I checked that ChunkImpl class sets all these values. I am looking into the correct
code? If yes, I can use relevant key value pair from here.
>>
>> Please Suggest
>> Regards
>> Stuti
>>
>> -----Original Message-----
>> From: Bill Graham [mailto:billgraham@gmail.com]
>> Sent: Friday, October 01, 2010 9:55 PM
>> To: chukwa-user@incubator.apache.org
>> Subject: Re: How to use metadata sent by agent to collector
>>
>> Can you be more specific regarding what you're trying to do? The
>> ServlerCollector, Demux and Processor classes all have access to the
>> Chunk object which encapsulated the fields you list, so it's not clear
>> what field access you're missing.
>>
>> On Fri, Oct 1, 2010 at 6:15 AM, Stuti Awasthi
>> <Stuti_Awasthi@persistent.co.in> wrote:
>>> Hi all,
>>>
>>>
>>>
>>> I noticed that when agent sends chunks to collector it also sends the
>>> metadata along with it, but in demux.java class , we are just dealing with
>>> the record entry that comes from agent.
>>>
>>> How can I use this metadata information that is sent by agent to collector
>>> also on collector side.
>>>
>>>
>>>
>>> Example , each chunk sends the meta data given as below , how can I use
>>> other fields like Key  or capp on my collector side.
>>>
>>> {
>>>
>>>     "DataType": "logtype",
>>>
>>>     "Key":
>>> "1285916400000/iaas-1-0-16-3e-45-46-7d.iaas.torolab.ibm.com/1285917710741",
>>>
>>>     "Timestamp": 1285917710741,
>>>
>>>     "mapFields": {
>>>
>>>   "capp": "/home/hadoop/xend.log",
>>>
>>>   "csource": "iaas-1-0-16-3e-45-46-7d.iaas.torolab.ibm.com",
>>>
>>>   "ctags": " cluster=\"chukwa",
>>>
>>>  "body" : "xyz"
>>>
>>> }
>>>
>>>
>>>
>>> Any pointers will be helpful.
>>>
>>>
>>>
>>> Regards
>>>
>>> Stuti Awasthi
>>>
>>> DISCLAIMER ========== This e-mail may contain privileged and confidential
>>> information which is the property of Persistent Systems Ltd. It is intended
>>> only for the use of the individual or entity to which it is addressed. If
>>> you are not the intended recipient, you are not authorized to read, retain,
>>> copy, print, distribute or use this message. If you have received this
>>> communication in error, please notify the sender and delete all copies of
>>> this message. Persistent Systems Ltd. does not accept any liability for
>>> virus infected mails.
>>
>> DISCLAIMER
>> ==========
>> This e-mail may contain privileged and confidential information which is the property
of Persistent Systems Ltd. It is intended only for the use of the individual or entity to
which it is addressed. If you are not the intended recipient, you are not authorized to read,
retain, copy, print, distribute or use this message. If you have received this communication
in error, please notify the sender and delete all copies of this message. Persistent Systems
Ltd. does not accept any liability for virus infected mails.
>>
>
> DISCLAIMER
> ==========
> This e-mail may contain privileged and confidential information which is the property
of Persistent Systems Ltd. It is intended only for the use of the individual or entity to
which it is addressed. If you are not the intended recipient, you are not authorized to read,
retain, copy, print, distribute or use this message. If you have received this communication
in error, please notify the sender and delete all copies of this message. Persistent Systems
Ltd. does not accept any liability for virus infected mails.
>

Mime
View raw message