incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Leo Neumeyer (Commented) (JIRA)" <>
Subject [jira] [Commented] (S4-21) Create standard PEs, and Event classes
Date Fri, 11 Nov 2011 19:14:51 GMT


Leo Neumeyer commented on S4-21:

Makes sense to have a generic stream, I think. 

We could have a factory method for the generic stream that only takes these arguments:

- stream name
- key finder class -or- key finder string
- target PE

(finder string is in the todo list, we already have it in 0.4. the string is parsed to generate
the finder class. It's an option for developers who dont' want to write key finder classes.)

Another thought is, should the Event base class be the generic event class? If the map is
not created the overhead would be limited to one reference field. The benefit is that all
events will be augmentable without having to subclass.

Why would we do this? because it provides an easy way to add meta-data or annotations to events.
For example, for 2-way communication we need to send the origin address in the event over
several hops. In NLP, we may want to add annotations to the event as it traverses several
PEs. The annotator chain may change often, making it impractical to use dedicated event classes.
The Map in base Event will make it possible to append these type of info using a simple pattern.
(Of course this is more costly than using event classes but requires less boilerplate and
fewer classes.) This design provide flexibility to use classes for static typing and speed
or maps for flexibility or a combination of both.

> Create standard PEs, and Event classes
> --------------------------------------
>                 Key: S4-21
>                 URL:
>             Project: Apache S4
>          Issue Type: Improvement
>    Affects Versions: 0.5
>            Reporter: Leo Neumeyer
>            Priority: Minor
>             Fix For: 0.5
> We should provide some reusable classes such as JoinPE and a MapEvent. The MapEvent is
a schema-less reusable event class.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message