streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From steveblackmon <>
Subject [GitHub] incubator-streams pull request: Streams 222
Date Fri, 21 Nov 2014 21:19:41 GMT
GitHub user steveblackmon opened a pull request:

    Streams 222

    I implemented this requirement with a familiar technology well-suited for API applications
- creating streams-runtime-dropwizard.  Also refactored DatasiftPushProvider to run using
this provider, eliminating the need for any overrides or other hacks in the implementation.
    The runtime contains three endpoints: /streams/webhooks/json, /streams/webhooks/json_new_line,
and /streams/webhooks/json_meta which with pass any json documents posted to them to a StreamProvider
named GenericWebhookResouce.
    Additionally any other StreamProviders with @Path annotation will be added as jersey resources,
and registered as perpetual streams which pass documents to queues named for their respective
classes.  They can pick whatever path they want to bind.  Using the yaml file used at dropwizard
start-up, any DW/Jersey parameter including port can be set, and additional configuration
for the stream can be supplied.  DW and typesafe don't play nice together, so this is the
recommended way of supplying configuration to this runtime. modules dropwizard-webhook-console
and dropwizard-webhook-elasticsearch are working examples of how to use this runtime.
    I think the ability to build and run streams which expose webhooks is a critical feature
gap this PR addresses, in a relatively flexible manner.  I look forward to your feedback.

You can merge this pull request into a Git repository by running:

    $ git pull STREAMS-222

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #141
commit a57c017211a499372fda1a511b6a7db063360f04
Author: sblackmon <>
Date:   2014-11-19T18:54:57Z

    drop wizard runtime

commit e6112906772aa5909e09f7246005e0caef5afff5
Author: sblackmon <>
Date:   2014-11-19T22:26:18Z

    Merge remote-tracking branch 'apache/master' into dropwizard

commit dbda9ed7075d0c65f0cd3c1db768dc35e63b22c0
Author: sblackmon <>
Date:   2014-11-20T21:28:31Z

    /streams/webhooks/* confirmed working

commit dbb69526e7469f6b988e9ed35793616d988102f6
Author: sblackmon <>
Date:   2014-11-20T21:30:15Z

    refactored DatasiftProvider to Resource

commit 1987427800b0cf90c197342ce522bab4cd72d605
Author: sblackmon <>
Date:   2014-11-20T23:18:42Z

    useful to have this metadata downstream

commit 069969d7f9496fbadc18a847648eead27396518e
Author: sblackmon <>
Date:   2014-11-20T23:19:39Z

    useful to have this metadata downstream

commit 6c989cb69ea0e2013d1543e987ce6b7325e30859
Author: sblackmon <>
Date:   2014-11-21T19:24:45Z

    added tests
    decided not to add StreamsResource to core at this time

commit 3d5f291a5f66f78d8728db0baf5ab6103ada42bf
Author: sblackmon <>
Date:   2014-11-21T19:26:43Z

    more tests
    javadoc headers
    refactor DatasiftPushProvider for compatibility

commit 149eb23109ce4a77042c58f65ee79eee786ac9f3
Author: sblackmon <>
Date:   2014-11-21T19:30:44Z



If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message