Return-Path: X-Original-To: apmail-streams-commits-archive@minotaur.apache.org Delivered-To: apmail-streams-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6366A11095 for ; Mon, 22 Sep 2014 15:48:20 +0000 (UTC) Received: (qmail 9863 invoked by uid 500); 22 Sep 2014 15:48:20 -0000 Delivered-To: apmail-streams-commits-archive@streams.apache.org Received: (qmail 9843 invoked by uid 500); 22 Sep 2014 15:48:20 -0000 Mailing-List: contact commits-help@streams.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@streams.incubator.apache.org Delivered-To: mailing list commits@streams.incubator.apache.org Received: (qmail 9834 invoked by uid 99); 22 Sep 2014 15:48:20 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 22 Sep 2014 15:48:20 +0000 X-ASF-Spam-Status: No, hits=-2000.8 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Mon, 22 Sep 2014 15:47:57 +0000 Received: (qmail 9509 invoked by uid 99); 22 Sep 2014 15:47:55 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 22 Sep 2014 15:47:55 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 4B0E49D126B; Mon, 22 Sep 2014 15:47:53 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: mfranklin@apache.org To: commits@streams.incubator.apache.org Date: Mon, 22 Sep 2014 15:47:53 -0000 Message-Id: X-Mailer: ASF-Git Admin Mailer Subject: [1/3] git commit: Merge pull request #4 from apache/master X-Virus-Checked: Checked by ClamAV on apache.org Repository: incubator-streams Updated Branches: refs/heads/master dc5e7de3c -> 2397f4b48 Merge pull request #4 from apache/master Merge Apache Master Project: http://git-wip-us.apache.org/repos/asf/incubator-streams/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-streams/commit/507e6796 Tree: http://git-wip-us.apache.org/repos/asf/incubator-streams/tree/507e6796 Diff: http://git-wip-us.apache.org/repos/asf/incubator-streams/diff/507e6796 Branch: refs/heads/master Commit: 507e67962e97e5e6b4c1de0bc0789c25804cd6af Parents: dca6475 dc5e7de Author: Ryan Ebanks Authored: Fri Sep 19 11:03:37 2014 -0500 Committer: Ryan Ebanks Committed: Fri Sep 19 11:03:37 2014 -0500 ---------------------------------------------------------------------- README.txt | 2 +- pom.xml | 2 +- streams-contrib/pom.xml | 1 + .../streams/console/ConsolePersistWriter.java | 5 +- .../ElasticsearchConfigurator.java | 41 +- .../ElasticsearchPersistDeleter.java | 34 +- .../ElasticsearchPersistUpdater.java | 66 +- .../ElasticsearchPersistWriter.java | 65 +- .../ElasticsearchWriterConfiguration.json | 6 +- .../streams-processor-jackson/pom.xml | 86 ++ .../CleanAdditionalPropertiesProcessor.java | 62 + .../regex/AbstractRegexExtensionExtractor.java | 13 +- .../org/apache/streams/urls/LinkResolver.java | 6 + .../streams-provider-datasift/pom.xml | 21 +- .../streams/datasift/csdl/DatasiftCsdlUtil.java | 6 +- .../DatasiftTypeConverterProcessor.java | 154 +++ .../provider/DatasiftManagedSourceSetup.java | 76 ++ .../provider/DatasiftStreamConfigurator.java | 15 +- .../DatasiftTypeConverterProcessor.java | 167 --- .../serializer/DatasiftActivitySerializer.java | 5 +- .../DatasiftDefaultActivitySerializer.java | 13 +- .../DatasiftInstagramActivitySerializer.java | 118 ++ .../DatasiftTweetActivitySerializer.java | 6 + .../main/jsonschema/com/datasift/Datasift.json | 1103 ------------------ .../com/datasift/DatasiftConfiguration.json | 22 - .../com/datasift/DatasiftPushConfiguration.json | 17 - .../datasift/DatasiftStreamConfiguration.json | 17 - .../com/datasift/DatasiftTwitterUser.json | 68 -- .../org/apache/streams/datasift/Datasift.json | 458 ++++++++ .../streams/datasift/DatasiftConfiguration.json | 133 +++ .../datasift/DatasiftPushConfiguration.json | 17 + .../datasift/DatasiftStreamConfiguration.json | 17 + .../datasift/facebook/DatasiftFacebook.json | 120 ++ .../datasift/instagram/DatasiftInstagram.json | 178 +++ .../interaction/DatasiftInteraction.json | 92 ++ .../datasift/twitter/DatasiftTwitter.json | 365 ++++++ .../datasift/twitter/DatasiftTwitterMedia.json | 126 ++ .../datasift/twitter/DatasiftTwitterUser.json | 68 ++ .../DatasiftTypeConverterProcessorTest.java | 5 +- .../DatasiftActivitySerializerTest.java | 19 +- .../test/resources/instagram_datasift_json.txt | 25 + .../metadata/facebook_post.mup | 694 +++++++++++ .../metadata/facebook_post.png | Bin 0 -> 1038039 bytes .../api/FacebookPostActivitySerializer.java | 29 +- .../processor/FacebookTypeConverter.java | 12 +- .../pagefeed/FacebookDataCollector.java | 138 +++ .../pagefeed/FacebookPageFeedDataCollector.java | 124 ++ .../pagefeed/FacebookPageFeedProvider.java | 28 + .../provider/pagefeed/FacebookProvider.java | 139 +++ .../FacebookStreamsPostSerializer.java | 60 + .../streams/facebook/FacebookConfiguration.json | 33 + .../provider/pagefeed/TestFacebookProvider.java | 94 ++ .../processor/InstagramTypeConverter.java | 19 +- .../provider/InstagramAbstractProvider.java | 196 ++++ .../provider/InstagramDataCollector.java | 145 +++ .../provider/InstagramRecentMediaCollector.java | 171 --- .../provider/InstagramRecentMediaProvider.java | 201 ---- .../InstagramRecentMediaCollector.java | 111 ++ .../InstagramRecentMediaProvider.java | 60 + .../userinfo/InstagramUserInfoCollector.java | 92 ++ .../userinfo/InstagramUserInfoProvider.java | 39 + .../serializer/InstagramUserInfoSerializer.java | 83 ++ .../InstagramRecentMediaCollectorTest.java | 159 --- .../InstagramRecentMediaProviderTest.java | 175 --- .../InstagramRecentMediaCollectorTest.java | 156 +++ .../InstagramRecentMediaProviderTest.java | 174 +++ .../InstagramUserInfoCollectorTest.java | 120 ++ .../apache/streams/data/util/MoreoverUtils.java | 2 +- .../SysomosBeatActivityConverter.java | 7 +- .../FetchAndReplaceTwitterProcessor.java | 4 +- .../twitter/provider/TwitterConfigurator.java | 70 ++ .../provider/TwitterStreamConfigurator.java | 99 -- .../twitter/provider/TwitterStreamProvider.java | 2 +- .../provider/TwitterTimelineProvider.java | 8 +- .../TwitterUserInformationProvider.java | 4 +- .../apache/streams/core/util/DatumUtils.java | 49 + streams-pojo/metadata/Serializer.mup | 371 ++++++ streams-pojo/metadata/serializer.png | Bin 0 -> 604337 bytes .../org/apache/streams/data/util/JsonUtil.java | 8 +- .../jackson/StreamsDateTimeSerializer.java | 4 - .../streams/jackson/StreamsJacksonMapper.java | 10 - .../jackson/StreamsPeriodDeserializer.java | 1 - .../jackson/StreamsPeriodSerializer.java | 2 - .../org/apache/streams/pojo/json/activity.json | 3 +- .../data/data/util/DateTimeSerDeTest.java | 2 - streams-runtimes/streams-runtime-local/pom.xml | 6 + .../local/tasks/StreamsPersistWriterTask.java | 2 + .../local/tasks/StreamsProcessorTask.java | 6 +- .../local/tasks/StreamsProviderTask.java | 19 +- .../local/builders/LocalStreamBuilderTest.java | 37 +- .../local/builders/ToyLocalBuilderExample.java | 6 +- .../streams/local/tasks/BasicTasksTest.java | 6 +- .../local/tasks/StreamsProviderTaskTest.java | 147 +++ .../test/processors/DoNothingProcessor.java | 2 +- .../PassthroughDatumCounterProcessor.java | 2 +- .../test/providers/NumericMessageProvider.java | 2 +- .../local/test/writer/DatumCounterWriter.java | 2 +- .../local/test/writer/DoNothingWriter.java | 2 +- .../local/test/writer/SystemOutWriter.java | 2 +- .../apache/streams/util/SerializationUtil.java | 4 +- 100 files changed, 5544 insertions(+), 2419 deletions(-) ----------------------------------------------------------------------