chukwa-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jerome Boulon (JIRA)" <>
Subject [jira] Commented: (CHUKWA-366) Custom Tags
Date Tue, 04 Aug 2009 18:24:14 GMT


Jerome Boulon commented on CHUKWA-366:

I would need more information because it's already possible to add some tags directly from
chukwa-agent.xml or dynamically.
So let me know if you need something else.

    <description>The cluster's name for this agent</description>

At the beginning the property name was chukwaAgent.cluster but I changed it to be able to
define some additional tags, cluster is just one tag that is there by default.

So if you want to define a role for example you could do:

    <value>cluster="@TODO-CLUSTER-NAME@" role="Database"</value>
    <description>The cluster's name for this agent and his role</description>

Also, there's a public API that you can use to set additional tags and the FileAdaptor is
using it to add a timestamp. This is done by this line: chunk.addTag("time=\"" + fileTime
+ "\"");

- org.apache.hadoop.chukwa.Chunk#getTag(String) has not been implemented because we coudln't
agree on a standard format for a tag.
So I would rather like to move that method to an helper class unless everybody agree on this
format: tagname="val". Personally I'm ok with this format since I'm the one that put it in

That's being said, I'm currently working on something else that may impact the Chunk class.
So here more information on what I'm doing.
I'm working on a streaming implementation for Chukwa and I need to manage the chunk  routing
dynamically using some rules. 
The rules are simple string comparison against some specific fields like host, category, cluster
and something similar to the role.
So I could either use the current Map or tags. The Map is not necessary the best option since
I will add some fields that are more like dimensions/metadata (cluster,host,rack, role,etc)
and are not part of the data. Tags will be the right place to do that but there's an overhead
in using only text.

So my preference will be to add another Map for storing dimensions or metadata related to
the list of records. This will simplify all the coding, speedup the code because we will no
longer use regex to match some tags but will add some additional bytes.

And in between, we could decide to use the tag field but with a json string.

> Custom Tags
> -----------
>                 Key: CHUKWA-366
>                 URL:
>             Project: Hadoop Chukwa
>          Issue Type: New Feature
>          Components: User Interface
>    Affects Versions: 0.3.0
>         Environment: Web Stack
>            Reporter: Aaron Beitch
>            Assignee: Ari Rabkin
>             Fix For: 0.3.0
>         Attachments: generaltagging.patch
> Being able to specify custom tags for a machine.
> I would like to be able to tag a machine as part of a specific stack, such as Stack1
and Stack2, which is already implemented as the 'cluster' tag specified in chukwa-agent-conf.xml.
I would also like to be able to tag each machine according to its role, such as WebServer
or DataBase. This is not currently doable. The best solution would probably be to allow arbitrary
tags. This functionality should be exposed also in the use of, such that I can filter
on any tag.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message