Return-Path: X-Original-To: apmail-streams-commits-archive@minotaur.apache.org Delivered-To: apmail-streams-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9B90310D68 for ; Wed, 19 Nov 2014 20:33:02 +0000 (UTC) Received: (qmail 7861 invoked by uid 500); 19 Nov 2014 20:33:02 -0000 Delivered-To: apmail-streams-commits-archive@streams.apache.org Received: (qmail 7842 invoked by uid 500); 19 Nov 2014 20:33:02 -0000 Mailing-List: contact commits-help@streams.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@streams.incubator.apache.org Delivered-To: mailing list commits@streams.incubator.apache.org Received: (qmail 7833 invoked by uid 99); 19 Nov 2014 20:33:02 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 19 Nov 2014 20:33:02 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED,T_RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.3] (HELO mail.apache.org) (140.211.11.3) by apache.org (qpsmtpd/0.29) with SMTP; Wed, 19 Nov 2014 20:33:00 +0000 Received: (qmail 7549 invoked by uid 99); 19 Nov 2014 20:32:40 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 19 Nov 2014 20:32:40 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 643CF9452CB; Wed, 19 Nov 2014 20:32:40 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: sblackmon@apache.org To: commits@streams.incubator.apache.org Date: Wed, 19 Nov 2014 20:32:40 -0000 Message-Id: <259d870455e4475c9e29e28877999744@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [1/5] incubator-streams git commit: Merge pull request #13 from apache/master X-Virus-Checked: Checked by ClamAV on apache.org Repository: incubator-streams Updated Branches: refs/heads/master 56011e4b6 -> 590b053b9 Merge pull request #13 from apache/master Merge Apache Master 2014/11/17 Project: http://git-wip-us.apache.org/repos/asf/incubator-streams/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-streams/commit/f3de6f88 Tree: http://git-wip-us.apache.org/repos/asf/incubator-streams/tree/f3de6f88 Diff: http://git-wip-us.apache.org/repos/asf/incubator-streams/diff/f3de6f88 Branch: refs/heads/master Commit: f3de6f8847f2091f68792e90a9643fd541f4e0dc Parents: 76763f8 9b89c08 Author: Ryan Ebanks Authored: Mon Nov 17 10:32:38 2014 -0600 Committer: Ryan Ebanks Committed: Mon Nov 17 10:32:38 2014 -0600 ---------------------------------------------------------------------- pom.xml | 61 +++++ streams-components/pom.xml | 62 +++++ streams-components/streams-http/README.md | 16 ++ streams-components/streams-http/pom.xml | 153 +++++++++++ .../components/http/HttpConfigurator.java | 62 +++++ .../http/processor/SimpleHTTPGetProcessor.java | 268 ++++++++++++++++++ .../http/provider/SimpleHTTPGetProvider.java | 230 ++++++++++++++++ .../components/http/HttpConfiguration.json | 50 ++++ .../http/HttpProcessorConfiguration.json | 28 ++ .../http/HttpProviderConfiguration.json | 18 ++ .../main/jsonschema/StreamsConfiguration.json | 3 + streams-contrib/pom.xml | 5 +- streams-contrib/streams-persist-console/pom.xml | 34 +++ .../streams/console/ConsolePersistReader.java | 11 +- .../streams/console/ConsolePersistWriter.java | 11 +- .../streams/jackson/TypeConverterProcessor.java | 112 ++++++++ .../test/TypeConverterProcessorTest.java | 95 +++++++ .../streams-processor-peoplepattern/pom.xml | 138 ++++++++++ .../peoplepattern/AccountTypeProcessor.java | 75 +++++ .../peoplepattern/DemographicsProcessor.java | 76 ++++++ .../streams/peoplepattern/AccountType.json | 27 ++ .../streams/peoplepattern/Demographics.json | 60 ++++ .../resources/templates/peoplepatternactor.json | 25 ++ .../DatasiftActivitySerializerProcessor.java | 89 ++++++ .../DatasiftTypeConverterProcessor.java | 7 +- .../serializer/DatasiftActivitySerializer.java | 12 +- .../DatasiftDefaultActivitySerializer.java | 214 --------------- .../serializer/DatasiftEventClassifier.java | 53 ++++ .../DatasiftInstagramActivitySerializer.java | 8 +- .../DatasiftInteractionActivitySerializer.java | 222 +++++++++++++++ .../DatasiftTweetActivitySerializer.java | 266 ------------------ .../DatasiftTwitterActivitySerializer.java | 272 +++++++++++++++++++ .../datasift/util/StreamsDatasiftMapper.java | 11 +- .../apache/streams/datasift/DatasiftPush.json | 30 ++ .../com/datasift/test/DatasiftSerDeTest.java | 18 +- .../DatasiftTypeConverterProcessorTest.java | 72 ----- .../DatasiftActivitySerializerTest.java | 82 +++--- .../serializer/DatasiftEventClassifierTest.java | 64 +++++ ...DatasiftInstagramActivitySerializerTest.java | 43 +++ ...tasiftInteractionActivitySerializerTest.java | 48 ++++ .../DatasiftTwitterActivitySerializerTest.java | 43 +++ .../api/FacebookPostActivitySerializer.java | 244 +---------------- .../serializer/FacebookActivityUtil.java | 153 ++++++++++- .../facebook/test/FacebookPostSerDeTest.java | 28 +- .../google-gplus/pom.xml | 32 +++ .../processor/GooglePlusCommentProcessor.java | 87 ++++++ .../gplus/provider/AbstractGPlusProvider.java | 234 ++++++++++++++++ .../gplus/provider/GPlusActivitySerializer.java | 4 +- .../gplus/provider/GPlusDataCollector.java | 50 ++++ .../provider/GPlusHistoryProviderTask.java | 106 -------- .../google/gplus/provider/GPlusProvider.java | 189 ------------- .../provider/GPlusUserActivityCollector.java | 108 ++++++++ .../provider/GPlusUserActivityProvider.java | 18 ++ .../gplus/provider/GPlusUserDataCollector.java | 79 ++++++ .../gplus/provider/GPlusUserDataProvider.java | 18 ++ .../util/GPlusCommentDeserializer.java | 98 +++++++ .../serializer/util/GooglePlusActivityUtil.java | 50 ++++ .../com/google/gplus/GPlusConfiguration.json | 44 ++- .../gplus/GooglePlusCommentSerDeTest.java | 114 ++++++++ .../provider/TestAbstractGPlusProvider.java | 82 ++++++ .../TestGPlusUserActivityCollector.java | 268 ++++++++++++++++++ .../provider/TestGPlusUserDataCollector.java | 131 +++++++++ .../resources/google_plus_comments_jsons.txt | 3 + streams-contrib/streams-provider-google/pom.xml | 6 + .../rss/provider/RssStreamProviderTest.java | 6 +- .../streams-provider-twitter/pom.xml | 11 + .../processor/TwitterEventProcessor.java | 194 ------------- .../twitter/processor/TwitterTypeConverter.java | 209 -------------- .../processor/TwitterUrlApiProcessor.java | 73 +++++ .../provider/TwitterEventClassifier.java | 58 ++-- .../provider/TwitterTimelineProvider.java | 7 +- .../serializer/StreamsTwitterMapper.java | 11 +- .../TwitterJsonActivitySerializer.java | 24 +- .../TwitterJsonDeleteActivitySerializer.java | 6 + .../TwitterJsonRetweetActivitySerializer.java | 6 + .../TwitterJsonTweetActivitySerializer.java | 6 + .../TwitterJsonUserActivitySerializer.java | 6 + ...erJsonUserstreameventActivitySerializer.java | 6 + .../streams/twitter/test/SimpleTweetTest.java | 11 +- .../twitter/test/TweetActivitySerDeTest.java | 6 +- .../streams/twitter/test/TweetSerDeTest.java | 6 +- .../test/TwitterEventClassifierTest.java | 34 +++ streams-monitoring/pom.xml | 109 ++++++++ .../jackson/DatumStatusCounterDeserializer.java | 76 ++++++ .../jackson/MemoryUsageDeserializer.java | 79 ++++++ .../jackson/StreamsTaskCounterDeserializer.java | 88 ++++++ .../jackson/ThroughputQueueDeserializer.java | 87 ++++++ .../monitoring/persist/MessagePersister.java | 34 +++ .../persist/impl/BroadcastMessagePersister.java | 94 +++++++ .../tasks/BroadcastMonitorThread.java | 171 ++++++++++++ .../org/apache/streams/pojo/json/Broadcast.json | 13 + .../pojo/json/DatumStatusCounterBroadcast.json | 22 ++ .../streams/pojo/json/MemoryUsageBroadcast.json | 30 ++ .../pojo/json/StreamsTaskCounterBroadcast.json | 38 +++ .../pojo/json/ThroughputQueueBroadcast.json | 38 +++ .../jackson/MemoryUsageDeserializerTest.java | 77 ++++++ .../impl/BroadcastMessagePersisterTest.java | 60 ++++ .../tasks/BroadcastMonitorThreadTest.java | 55 ++++ .../src/test/resources/MemoryUsageObjects.json | 1 + streams-pojo-extensions/pom.xml | 64 +++++ .../apache/streams/data/util/ExtensionUtil.java | 108 ++++++++ .../apache/streams/data/util/ActivityUtil.java | 14 +- .../jackson/StreamsDateTimeDeserializer.java | 23 +- .../streams/jackson/StreamsJacksonMapper.java | 20 ++ .../streams/jackson/StreamsJacksonModule.java | 9 + .../org/apache/streams/pojo/json/activity.json | 3 +- .../org/apache/streams/pojo/json/object.json | 2 +- streams-runtimes/streams-runtime-local/pom.xml | 5 + .../local/builders/LocalStreamBuilder.java | 19 +- .../streams/local/tasks/StreamsMergeTask.java | 7 + .../local/tasks/StreamsPersistWriterTask.java | 18 +- .../local/tasks/StreamsProcessorTask.java | 20 +- .../local/tasks/StreamsProviderTask.java | 15 + .../apache/streams/local/tasks/StreamsTask.java | 4 + .../local/builders/LocalStreamBuilderTest.java | 32 ++- .../streams/local/tasks/BasicTasksTest.java | 26 +- .../streams/pig/test/PigProcessDatumTest.java | 2 - .../pig/test/PigProcessDocumentTest.java | 2 - .../streams/pig/test/PigSerializerTest.java | 3 +- .../backoff/AbstractBackOffStrategy.java | 15 +- 120 files changed, 5800 insertions(+), 1683 deletions(-) ----------------------------------------------------------------------