streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Letourneau <>
Subject Re: Streams Subscriptions
Date Mon, 04 Feb 2013 15:03:10 GMT
I've modified the JSON a bit to account for these things - the filter
interface becomes very simple, expecting some string for query,
defines an evaluate method that returns boolean given an activity, and
the json specifies the @class - so an implementor could add a new osgi
bundle, implement the filter interface and tell streams to make a
subscription with the filter class specified (still some refining for
sure to go, but the latest is below):

    "authToken": "token",
    "filters": [
            "query": "string represented query of type expected by
filter implementation"
    "outputs": [
            "output_type": "http",
            "method": "post",
            "url": "",
            "delivery_frequency": "60",
            "max_size": "10485760",
            "auth_type": "none",
            "username": "username",
            "password": "password"

On Sat, Feb 2, 2013 at 5:39 PM, Jason Letourneau
<> wrote:
> Really great stuff - i will make sure filtering is componentized enough for this scenario
in my next commit - and start being good about jira tracking ;)
> Sent from my iPhone
> On Feb 1, 2013, at 5:22 PM, Craig McClanahan <> wrote:
>> On Fri, Feb 1, 2013 at 1:23 PM, Steve Blackmon [W2O Digital] <
>>> wrote:
>>> One nice thing lucene offer is support for nested conditional logic right
>>> in the query - so subscribers can request very complicated filters with a
>>> single filter tag in the json request.  Lucene is also the basis for
>>> querying elastic search, and some of the largest data providers such as
>>> Sysomos/Marketwire - within W2O we have a large library of lucene queries
>>> and it would be great to use those with minimal modification to configure
>>> streams.
>>> Lucene syntax makes sense to me ... I'd rather work on the more
>> "interesting" problems than designing a filter syntax :-).
>>> But this brings up a wider topic regarding adoption - many users will be
>>> migrating or integrating solutions where they filter based on lucene, or
>>> solr, or ham crest, or regex, etcŠ  So a plug-in architecture that would
>>> let users who can compile java embed whatever filtering logic works best
>>> for them into streams, without having to commit to master would be
>>> advisable.  Bonus points if those plugins can bring their own class path
>>> via osgi or similar approach.
>>> So, a "filter" would become just a set of Lucene (by default) search
>> expressions as strings, with pluggability for how the strings actually get
>> interpreted?  I like it.
>>> Steve Blackmon
>>> Director, Data Sciences
>>> 101 W. 6th Street
>>> Austin, Texas 78701
>>> cell 512.965.0451 | work 512.402.6366
>>> twitter @steveblackmon
>>> Craig

View raw message