streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From steveblackmon <>
Subject [GitHub] incubator-streams pull request: Streams 168
Date Tue, 14 Oct 2014 21:34:32 GMT
GitHub user steveblackmon opened a pull request:

    Streams 168

    There's quite a bit going on in this PR, so here's a summary:
    To date, modules in streams-contrib have implemented interfaces from streams-core.  Code
re-use has been intra-contrib-module, or occasionally cross-contrib-module, with a few helper
classes in streams-pojo playing a supporting role.
    This PR introduces a new top-level sub-module, streams-components, where I've placed streams-http.
 streams-http contains a provider and a processor, both of which can be employed in a stream
to collect json documents from remote REST endpoints.  The config properties of streams-http
were chosen to align with camel's http support module, anticipating compatibility will be
desirable at some point in the future.
    In addition, the PR contains new processors (in streams-provider-twitter and streams-processor-peoplepattern)
which employ streams-http to collect data from specific data supplier endpoints, with a high
degree of code reuse from streams-http.  Essentially they just override a few methods but
do not need to introduce their configuration or alter any aspect of the REST call other than
parameters and headers.  I've built code that uses the provider as well.
    The JSON responses from the REST calls in these reference processors are then appended
to specific extensions fields (activity.extensions.X,, activity.object.extensions.X,
etc.. for arbitrary X, sited on the activity or below (on any sub-node whose field extends
ActivityObject.)  Utility classes supporting extensions are under streams-pojo-extensions.
 Modules that were importing these classes from streams-pojo would need to update to import
streams-pojo-extensions as well.
    Philosophically, future modules containing providers, processors, and persisters which
are both useful as standalone components in streams, and which are designed to simplify/harden
channel/API/database-specific components that extend them, would be placed under streams-components.
 This gives us a dedicated sub-partitioned module for valuable utility features (outside of
core), and will drastically curtail tendancies for one contrib module to explicitly require
    Lastly, I've built streams-example modules utilizing each of these processors, which are
pushed to w2ogroup/streams-examples under the STREAMS-168 branch.
    I look forward to your feedback on this proposed reorg.

You can merge this pull request into a Git repository by running:

    $ git pull STREAMS-168

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #104
commit 1464819fd7ea147830a4211ea2ac6af4aa715022
Author: sblackmon <>
Date:   2014-09-14T15:24:47Z

    just pons

commit 34232ad87b50253913fba0dabb6fa9af591f388d
Author: sblackmon <>
Date:   2014-09-15T18:50:32Z

    added streams-components module
    added streams-processor-http module
    activated streams-pojo-extensions module
    tweaks to pojo, require id and remove non-sensical defaults

commit b8ccf9f6869c64f1f6e25f668942ea0de9fec2eb
Author: sblackmon <>
Date:   2014-09-15T23:38:09Z

    added HttpConfigurator

commit a576460911464cf260e37886ce840f4e57dd5f30
Author: sblackmon <>
Date:   2014-09-16T19:24:53Z

    class need not be abstract
    rootDocument is what should exit processor

commit d62061dec8628484e66d3494e02f1f65efafda41
Author: Steve Blackmon <>
Date:   2014-10-12T22:49:01Z

    introduced a SimpleHTTPGetProvider
    reorganized configuration
    added access_token support
    added http basic auth support

commit 9a575322231e4a8a69c56f06da743ffe3211ccb4
Author: Steve Blackmon <>
Date:   2014-10-13T01:20:52Z

    committing PeoplePattern processor

commit ad5f90cc115c40e0ef480a85957bbb4c98157cf6
Author: Steve Blackmon <>
Date:   2014-10-13T01:21:22Z

    removing validation, as jackson does not contain a default implementation until 2.5 release

commit 0c8e67ce26e131282aaaa03815ad6daefd684346
Author: Steve Blackmon <>
Date:   2014-10-13T04:13:29Z

    a few package name changes
    working twitterurlapiprocess which uses streams-http (w/o authentication)
    working people pattern processors which use streams-http (w/ authentication)

commit d2ffe62baf88618c7c8a6b9a1e5a79d21ed9e3fd
Author: sblackmon <>
Date:   2014-10-13T22:07:19Z

    added license header and javadoc comments


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message