streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renato Marroquín Mogrovejo <renatoj.marroq...@gmail.com>
Subject Re: Starting the project
Date Thu, 22 May 2014 12:22:47 GMT
Maybe even something like this should be done but for the contrib modules.

https://github.com/apache/incubator-streams/blob/master/streams-contrib/streams-provider-rss/src/test/java/org/apache/streams/rss/test/Top100FeedsTest.java

That would be the something like examples. Wdyt?


Renato M.


2014-05-22 12:05 GMT+02:00 Renato Marroquín Mogrovejo <
renatoj.marroquin@gmail.com>:

> Hi Steve,
>
> Comments inline.
>
> 2014-05-21 20:59 GMT+02:00 Steve Blackmon <sblackmon@apache.org>:
>
> I will happily accept pull requests to streams-examples, although I
>> don't think it would need to change for an update to the provider to
>> be applied (except perhaps to add that verb filtering step)
>>
>
> This is what I meant.
>
>
>> I am willing to donate all of the examples, but I don't think they
>> belong in the main repo.  What if there were 50 examples many
>> containing integration tests?  Build would take forever - plus I don't
>> think that a single example failing a test should break the build.
>> Mahout suffered from really long-builds and handled it by separating
>> the examples out from core.
>>
>
> Well, over in Apache Giraph they have over 20 graph example algorithms and
> the way they test them is just by doing some simple computations, I don't
> know how long it takes, but I think it makes sense to at least try it and
> see how long it takes.
> Over in Gora we have a test suite with the different operations supported
> that is run over the different data stores, and depending on your machine,
> it might take from 10 - 20 mins. There was a time where the HBase tests
> were taking around 30 mins by themselves but we fixed them and are now run
> in less than 5 mins.
> Maybe your examples could be converted into a test suite? I mean reading
> from twitter(or a file) and persisting into different places. But anyhow,
> your examples should be referenced somewhere as it is what it makes sense
> for a newcomer :D
>
> When the project was hosting in SVN, we had the ability to create and
>> checkout different roots (the master pom was in a different root for
>> example).  If we can do the same using Git I propose we put the
>> examples in as a flat multi-module project that way.
>>
>
> +1
>
>
> Renato M.
>
> Any thoughts?
>>
>> Steve Blackmon
>> sblackmon@apache.org
>>
>> On Wed, May 21, 2014 at 11:59 AM, Renato Marroquín Mogrovejo
>> <renatoj.marroquin@gmail.com> wrote:
>> > I tried updating the site through ApacheCMS but I guess I don't have the
>> > rights to do it ;)
>> > This is what I got:
>> >
>> > The operation is forbidden by the server: Commit failed (details
>> follow)::
>> > Changing file
>> >
>> '/usr/local/cms/wc/streams/rmarroquin-NlOO6g/trunk/content/source-repository.mdtext'
>> > is forbidden by the server: Access to
>> >
>> '/repos/asf/!svn/txr/1596615-yz8u/incubator/streams/site/trunk/content/source-repository.mdtext'
>> > forbidden at /usr/local/cms/webgui/lib/ASF/CMS/WC/Commit.pm line 64
>> >
>> > So what is the best/easy way to update the site?
>> >
>> > @Steve
>> > Yeah, I finally got the time to try this out :) and looks very promising
>> > for sure!
>> > Is anybody working on STREAMS-57? Could I pick it up? I think this
>> should
>> > be pretty straight forward.
>> > And are there any plans to put your examples into the code base? It's
>> quite
>> > puzzling going from repo to another (plus the one from the site) to
>> > accomplish this. And then if I could work on STREAMS-57, would I have
>> to do
>> > the pull request into W2OGroup/streams-examples repo? I don't think so
>> > right? how would this work?
>> > Thanks!
>> >
>> >
>> > Renato M.
>> >
>> >
>> > 2014-05-21 17:42 GMT+02:00 Steve Blackmon <steve@blackmon.org>:
>> >
>> >> Cool I'm glad to hear you got it running. That exception occurs because
>> >> twitter sends a preamble listing all of your friends when you open a
>> >> stream. The provider doesn't have logic to handle it yet, so it tries
>> to
>> >> build an activity and throws that exception.
>> >>
>> >> STREAMS-57 would suppress this warning - although once non-content
>> >> activities with new verbs like follow are generated we will want a
>> >> filtering mechanism in place so it remains easy to build indexes of
>> just
>> >> shares and posts.
>> >>
>> >> On Wed, May 21, 2014 at 9:23 AM, Renato Marroquín Mogrovejo <
>> >> renatoj.marroquin@gmail.com> wrote:
>> >> > Hey Matt,
>> >> >
>> >> > So the examples worked :D at least the one retrieving the twitter
>> user's
>> >> > history. The one trying to track users activity keeps on getting this
>> >> > exception:
>> >> >
>> >> > 16:09:11.151 [pool-2-thread-2] DEBUG o.a.s.t.p.TwitterTypeConverter
-
>> >> > ACTIVITY
>> >> > org.apache.streams.exceptions.ActivitySerializerException:
>> unrecognized
>> >> type
>> >> > at
>> >> >
>> >>
>> >>
>> org.apache.streams.twitter.serializer.TwitterJsonActivitySerializer.deserialize(TwitterJsonActivitySerializer.java:74)
>> >> > at
>> >> >
>> >>
>> >>
>> org.apache.streams.twitter.processor.TwitterTypeConverter.convert(TwitterTypeConverter.java:65)
>> >> > at
>> >> >
>> >>
>> >>
>> org.apache.streams.twitter.processor.TwitterTypeConverter.process(TwitterTypeConverter.java:146)
>> >> > at
>> >> >
>> >>
>> >>
>> org.apache.streams.local.tasks.StreamsProcessorTask.run(StreamsProcessorTask.java:72)
>> >> > at
>> >> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>> >> > at java.util.concurrent.FutureTask.run(FutureTask.java:262)
>> >> > at
>> >> >
>> >>
>> >>
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
>> >> > at
>> >> >
>> >>
>> >>
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>> >> > at java.lang.Thread.run(Thread.java:744)
>> >> >
>> >> > But everything seems to work in spite of that exception :) Cool
>> stuff! So
>> >> > can I open a JIRA issue for updating the site and putting the patch
>> there
>> >> > then?
>> >> > I will write another email with some other issues.
>> >> > Thanks Matt!
>> >> >
>> >> >
>> >> > Renato M.
>> >> >
>> >> >
>> >> > 2014-05-21 15:59 GMT+02:00 Matt Franklin <m.ben.franklin@gmail.com>:
>> >> >
>> >> >> On Wed, May 21, 2014 at 9:00 AM, Renato Marroquín Mogrovejo <
>> >> >> renatoj.marroquin@gmail.com> wrote:
>> >> >>
>> >> >> > Hi Matt,
>> >> >> >
>> >> >> > Thank for pointing me to these examples. I will try them out
and
>> >> report
>> >> >> > back.
>> >> >> > But I have more questions now. I have found:
>> >> >> >
>> >> >> > https://github.com/apache/incubator-streams --> referenced
by the
>> >> >> examples
>> >> >> > you sent me and more active
>> >> >> > https://github.com/apache/streams --> referenced by the
website
>> >> >> >
>> >> >>
>> >> >> Oops. Yes it is the first one. The second one was a mirror of SVN.
>> >> >>
>> >> >>
>> >> >> >
>> >> >> > Which one is the official one? I guess the first one right.
>> Shouldn't
>> >> we
>> >> >> > change this on the website? does that mean that Apache Streams
>> have
>> >> two
>> >> >> git
>> >> >> > repos? I can fix that on the website if you point me to the
right
>> >> place.
>> >> >> >
>> >> >>
>> >> >> Website is in SVN using Apache CMS.
>> >> >>
>> >> >>
>> >> >> >
>> >> >> >
>> >> >> > Renato M.
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > 2014-05-21 2:14 GMT+02:00 Matt Franklin <m.ben.franklin@gmail.com
>> >:
>> >> >> >
>> >> >> > > On Tue, May 20, 2014 at 4:28 PM, Renato Marroquín Mogrovejo
<
>> >> >> > > renatoj.marroquin@gmail.com> wrote:
>> >> >> > >
>> >> >> > > > Sorry I pressed the sent button before I finished
writing the
>> >> email.
>> >> >> > > > Anyways . . . Tomcat 7.0.39, ServiceMix 5.0.1 and
Cassandra
>> 2.0.4.
>> >> >> > > > But I am getting this exception:
>> >> >> > > >
>> >> >> > >
>> >> >> > > We are unfortunately in the middle of converging the
new model
>> and
>> >> the
>> >> >> > old
>> >> >> > > model. This runtime should be working, but it needs some
work to
>> >> >> ensure
>> >> >> > > that the new components in streams-contrib work. If you
are
>> looking
>> >> to
>> >> >> > get
>> >> >> > > started with a newer runtime, streams-runtime-local is
a good
>> place.
>> >> >> > >
>> >> >> > > Steve posted a link to some examples he wrote for how
to get
>> >> started:
>> >> >> > >
>> >> >> > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> >>
>> https://github.com/w2ogroup/streams-examples/tree/master/twitter-history-elasticsearch
>> >> >> > >
>> >> >> > > We need to create some of our own examples (or ask him
kindly to
>> >> donate
>> >> >> > > some to us), but until the site is fixed, this is a good
start.
>> I
>> >> will
>> >> >> > > create an issue for the Caml runtime issue you found
here.
>> >> >> > >
>> >> >> > >
>> >> >> > > >
>> >> >> > > > May 20, 2014 10:15:29 PM
>> org.apache.catalina.startup.ContextConfig
>> >> >> > > > getServletContainerInitializer
>> >> >> > > > SEVERE: The ServletContentInitializer
>> >> >> > > > [org.springframework.web.SpringServletContainerInitializer]
>> could
>> >> not
>> >> >> > be
>> >> >> > > > created
>> >> >> > > > java.lang.ClassNotFoundException:
>> >> >> > > > org.springframework.web.SpringServletContainerInitializer
>> >> >> > > > at
>> >> >> > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> >>
>> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1713)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> >>
>> org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1558)
>> >> >> > > > at java.lang.Class.forName0(Native Method)
>> >> >> > > > at java.lang.Class.forName(Class.java:270)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> >>
>> org.apache.catalina.startup.ContextConfig.getServletContainerInitializer(ContextConfig.java:1659)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> >>
>> org.apache.catalina.startup.ContextConfig.processServletContainerInitializers(ContextConfig.java:1569)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> >>
>> org.apache.catalina.startup.ContextConfig.webConfig(ContextConfig.java:1277)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> >>
>> org.apache.catalina.startup.ContextConfig.configureStart(ContextConfig.java:878)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> >>
>> org.apache.catalina.startup.ContextConfig.lifecycleEvent(ContextConfig.java:369)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> >>
>> org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:119)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> >>
>> org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:90)
>> >> >> > > > at
>> >> >> > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> >>
>> org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5269)
>> >> >> > > > at
>> >> >> > > >
>> >> org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150)
>> >> >> > > >
>> >> >> > > > That let me to [2]. Is this related to this? or
is it just me
>> >> doing
>> >> >> > > things
>> >> >> > > > wrong here?
>> >> >> > > > Thanks for any pointers!
>> >> >> > > >
>> >> >> > > >
>> >> >> > > > Renato M.
>> >> >> > > >
>> >> >> > > > 2014-05-20 22:25 GMT+02:00 Renato Marroquín Mogrovejo
<
>> >> >> > > > renatoj.marroquin@gmail.com>:
>> >> >> > > >
>> >> >> > > > > Hey guys,
>> >> >> > > > >
>> >> >> > > > > This is a long overdue email but with little
steps I will
>> start
>> >> >> going
>> >> >> > > :)
>> >> >> > > > > So I have been trying to run the whole project
based on[1].
>> As
>> >> >> there
>> >> >> > is
>> >> >> > > > no
>> >> >> > > > > version for any of the pieces involved in running
this (if
>> there
>> >> is
>> >> >> > > > please
>> >> >> > > > > point me to them, if not, I am happy to put
them on the site
>> >> >> > > somewhere).
>> >> >> > > > I
>> >> >> > > > > chose
>> >> >> > > > >
>> >> >> > > > >
>> >> >> > > > >
>> >> >> > > > > [1] http://streams.incubator.apache.org/streams-usage.html
>> >> >> > > > > [2]
>> >> >> > > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>> >>
>> http://stackoverflow.com/questions/7692497/tomcat-wont-load-my-meta-inf-services-javax-servlet-servletcontainerinitializ
>> >> >> > > > >
>> >> >> > > >
>> >> >> > >
>> >> >> >
>> >> >>
>> >>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message