streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Letourneau <jletournea...@gmail.com>
Subject Re: Subscriber/ Publisher handling of activity
Date Wed, 10 Jul 2013 00:01:48 GMT
A clarifying point on the iteration - the aggregator service knows about
each subscriber and is responsible for pulling activities from the queue
and offering them to each subscriber - this is a potential bottleneck and
it would be good to get a discussion going on how to mitigate that

On Tuesday, July 9, 2013, Jason Letourneau wrote:

> The last discussion on this topic had subscribers applying a filter to
> each published message on the queue - there should be some stub classes in
> the source that shows this thought direction - each subscriber would be
> iterated over and asked to process each published activity on the queue -
> they would apply a filter adhering to the filter interface - the
> implementation of that filter could be anything - one thought was a dsl
> like lucene syntax could be the default implementation - to answer your
> foundational question - publishers should have no knowledge of who is
> subscribed an subscriber should be able to filter in the best way for them
> (I.e. based on source o message, user, activity streams properties etc)
>
> Jason
>
>
> On Tuesday, July 9, 2013, Danny Sullivan wrote:
>
>> Will publishers or subscribers be in charge or making sure that only
>> specific activity stream entries make it to a certain queue?
>> If publishers are in charge, I would imagine that there would exist a
>> list of all subscribers for each publisher. Then each activity published
>> would be added to all the subscribers in that publishers subscriber list.
>> If subscribers are in charge, each subscriber would have a list of
>> publishers he/she is subscribed to. Then on some sort of timer, the list
>> would be iterated through and all activity entires not already consumed by
>> that subscriber would be outputted.
>> Looking at the application architecture here:
>> http://streams.incubator.apache.org/architecture.html It looks like all
>> activity is passed through a single queue. If this is going to be the
>> implementation going forward, I would think it would make more sense for
>> subscribers to handle the filtering. That would make it so that all
>> activity entires could be dumped in a single database by the publishers and
>> activity could be extracted and filtered based on some list kept by each
>> individual subscriber. Let me know if this sounds like it aligns with the
>> direction of the project. I would like to have the functionality to allow
>> subscribers to get only specific messages that are published.
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message