streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Baxter <rbaxte...@gmail.com>
Subject Re: Question about processing architecture
Date Thu, 06 Jun 2013 12:39:57 GMT
On Thu, Jun 6, 2013 at 8:08 AM, Beth Lavender <lavender.beth@gmail.com> wrote:
> On Tue, Jun 4, 2013 at 8:32 AM, Matt Franklin <m.ben.franklin@gmail.com>wrote:
>
>> On Mon, Jun 3, 2013 at 10:52 PM, Jason Letourneau
>> <jletourneau80@gmail.com>wrote:
>>
>> > The current vision is that filters will be implemented agnostic to the
>> > overall processing architecture - there may be subscribers using lucene
>> dsl
>> > as part of the initial streams implemetation- but the interface won't
>> > dictate how a subscriber filters it's activities -
>>
>>
>> Maybe I am misunderstanding your statement.  In my mind, we really need
>> inbound and outbound data pipelines.  I don't think a simple outbound
>> filter can solve this easily.  I don't see how the system can do
>> de-duplication, supersession, aggregation, etc during the outbound phase.
>>  We will need to do a lot of processing before we hit an intermediate
>> persistence layer that can then be used by subscriber filters and query
>> endpoints.
>>
>> The pipeline components themselves should be pluggable and we just need a
>> series of workflow events that they can hook and do work against the
>> incoming data.
>>
>> Am I off base?
>>
>
> The rollup [1] reference is useful.  There is an implied set use cases
> given the context for "views that support roll-up".  Do we have a set of
> use cases documented (or that can be referenced) that would help drive
> where in the architecture the plug ins are needed?

Views are just a subset of your stream in Connections.  One view where
rollup is not used is the action required view.  This will show all
activities that the application deemed to need some type of action
performed by a user, for example an approval for something.  This
concept of "action required" is also an extension to the activity
entry that IBM came up with.  We don't "rollup" activity entries in
this view because there might be several actions you need to take on a
single object and we don't want to give the impression that there may
on by one by rolling them up.

>
>>
>>
>> > I don't know that we've
>> > figured out whether the subscriber delegate tells the aggregate its
>> filter
>> > via an interface or whether aggregator tells each subscriber about every
>> > activity and the subscriber filters - either way - you can implement
>> > filters however you want - provides they adhere to the common filter
>> > interface (which to my recollection is very simplistic)
>> >
>> > On Monday, June 3, 2013, Lavender, Beth A wrote:
>> >
>> > > Many of our current systems that will feed the integrated activity
>> stream
>> > > are noisy. For example, if I update a page 4 times in  5 minutes it
>> > > generates an activity for each one.   I want to be able to set rule for
>> > > discard the last n activities if they have the same actor, verb, and
>> > object
>> > >  in x time frame.
>> > >
>> > > This assumes a sub processor that detects the pattern and takes an
>> action
>> > > described in a rule. Where do rules and sub processors fit in this
>> > > architecture? Is anyone doing this in their existing systems?
>> > >
>> > >
>> >
>>

Mime
View raw message