incubator-chukwa-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bill Graham <billgra...@gmail.com>
Subject Re: HttpTriggerAction - configuring N objects
Date Thu, 22 Apr 2010 00:27:57 GMT
Thanks guys. Eric, I think I'm on a similar path to what you're suggesting
except I'm using a two-step config to specidy a.) the action classes, and
b.) their configs. Jerome, I considered that, but gets tricky with
multi-value sets (like headers). At some point we get into delimiter
overload.

To get to specifics, here's an example of what I currently have:

- A comma separated list of TriggerAction classes to be invoked upon
successful completion of a successful demux run. This is similar to the
current data loader pattern and can also be used wherever else we later want
to fire TriggerActions.

  <property>
    <name>chukwa.post.demux.success.action</name>

<value>org.apache.hadoop.chukwa.extraction.demux.HttpTriggerAction,some.other.TriggerAction</value>
  </property>

- Then when the HttpTriggerAction runs, it knows how to look for it's
configs in the way that we've been discussing. For the above action, the
trigger event name will be passed to TriggerAction as an enum. The enum
would have a name like 'postDemuxSuccess', as well as a pointer to the base
config string for the event, like 'chukwa.trigger.post.demux.success'.

- HttpTriggerAction would in this case then know to look for it's configs
under
chukwa.trigger.post.demux.success.*http *and look for values like this:

  <property>
    <name>chukwa.trigger.post.demux.success.http.1.url</name>
    <value>http://site.com/firstTrigger</value>
  </property>

so the syntax of the config key then is:

chukwa.[eventName].[actionNS].N.[actionKeys]

Although other actions are free to implement all parts below eventName
however makes the most sense for their needs.

thanks,
Bill

On Wed, Apr 21, 2010 at 4:50 PM, Jerome Boulon <jboulon@netflix.com> wrote:

>  Hi,
> You can have a root configuration key that will give the list of keys to
> look for:
> Or you can look for the existence of http.N key in the configuration
> object.
>
> Ex
> <property >
> <name>myConfig.eventName.list</name>
> <value>http.1, http.2, ..., http.x</value>
> </property>
>
>
> Can you clarify what the eventName is?
>
> /Jerome.
>
>
> On 4/21/10 3:31 PM, "Bill Graham" <billgraham@gmail.com> wrote:
>
> Hi,
>
> As a follow up to CHUKWA-477, I'm writing a class called HttpTriggerAction
> that can hit one or more URLs upon successful completion of a demux job. I'd
> like to contribute it back unless anyone objects. Anyway, I'm looking for
> feedback though on how to configure this object.
>
> The issue is that since the class can hit N urls, it needs N sets of
> key-value configurations. The hadoop configurations model is just name-value
> pairs though, so I'm kicking around ideas around the best way to handle
> this.
>
> Specifically, I need to configure values for url, an optional HTTP method
> (default is GET), an optional collection of HTTP headers and an optional
> post body. I was thinking of just making a convention where key values could
> be incremented like below, but wanted to see if there were better
> suggestions out there.
>
> chukwa.trigger.action.[eventName].http.1.url=http://site.com/firstTrigger
> chukwa.trigger.action.[eventName].http.1.headers=User-Agent:chukwa
>
> chukwa.trigger.action.[eventName].http.2.url=http://site.com/secondTrigger
> chukwa.trigger.action.[eventName].http.2.method=POST
>
> chukwa.trigger.action.[eventName].http.2.headers=User-Agent:chukwa,Accepts:text/plain
> chukwa.trigger.action.[eventName].http.2.body=Some post body to submit
> ....
> chukwa.trigger.action.[eventName].http.N.url=
> chukwa.trigger.action.[eventName].http.N.method=
> chukwa.trigger.action.[eventName].http.N.headers=
> chukwa.trigger.action.[eventName].http.N.body=
>
> Since the action could potentially be used by other types of events, the
> event name should be included. This implies that we should add an eventName
> field to the TriggerAction.execute method in CHUKWA-477.
>
> Thoughts?
>
> thanks,
> Bill
>
>
>
>

Mime
View raw message