Mailing-List: contact dev-help@streams.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@streams.incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of m.ben.franklin@gmail.com
 designates 209.85.215.44 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAFNO4HjVdxjYYkMMMDBwNMVPVe3Wi00TKoYmBzyRjiZwOu0Hyw@mail.gmail.com>
References: 
 <CAENpzYUE2bdZKqRfe81zoY+zchsjWO+Rb56NoB+1ws=PR25F+Q@mail.gmail.com>
	<CAFNO4Hi_Oxwm71zYMpWLt1QGuoXBi7bkoO8o4YGaw-ZdV6MGTg@mail.gmail.com>
	<CAENpzYVg5=dKsdeuq7eMgZHqL_5TSPGY--TcNs+ujKrg4nMjfA@mail.gmail.com>
	<CAFNO4HgQnR0-UowO=LwUgKXo0UQs0S-ZcEp1fP=vqj8d0epHFg@mail.gmail.com>
	<CAENpzYXK1F=LYmgcaAMEww2PYX83CH3nFK+Uzep=SdGpD5EPCw@mail.gmail.com>
	<CAL8syOZoN30bW-H+SrF0poU3zThOyAEGNgaaHhE=VJTOwCCSTg@mail.gmail.com>
	<CAFNO4HjVdxjYYkMMMDBwNMVPVe3Wi00TKoYmBzyRjiZwOu0Hyw@mail.gmail.com>
Date: Mon, 25 Mar 2013 18:00:50 -0400
Message-ID: 
 <CAENpzYWUw8bHw54yKamenv06WyQ5r-ufRY8SJbafJPYhboQc0g@mail.gmail.com>
Subject: Re: ApacheCon Recap & Next Steps
From: Matt Franklin <m.ben.franklin@gmail.com>
To: dev@streams.incubator.apache.org
Content-Type: text/plain; charset=ISO-8859-1

On Fri, Mar 15, 2013 at 12:48 PM, Chris Geer <chris@cxtsoftware.com> wrote:
> To try some things I went ahead and forked the GitHub repo copy and checked
> in some changes.
>
> https://github.com/geerzo/streams
>
> I've got it all working like before but at the moment it's not really setup
> for OSGI since it requires exporting private packages. I'll continue to
> work on it but let me know what you think.

So far, this looks good to me.  Did you want to submit a pull request or patch?

>
> Chris
>
> On Thu, Mar 14, 2013 at 8:07 AM, Jason Letourneau
> <jletourneau80@gmail.com>wrote:
>
>> +1
>>
>> On Thu, Mar 14, 2013 at 10:45 AM, Matt Franklin
>> <m.ben.franklin@gmail.com> wrote:
>> > On Thu, Mar 14, 2013 at 10:42 AM, Chris Geer <chris@cxtsoftware.com>
>> wrote:
>> >> On Thu, Mar 14, 2013 at 6:52 AM, Matt Franklin <
>> m.ben.franklin@gmail.com>wrote:
>> >>
>> >>> On Wed, Mar 13, 2013 at 3:20 PM, Chris Geer <chris@cxtsoftware.com>
>> wrote:
>> >>> > On Wed, Mar 13, 2013 at 8:41 AM, Matt Franklin <
>> m.ben.franklin@gmail.com
>> >>> >wrote:
>> >>> >
>> >>> >> For all of you who missed it, Craig gave a nice intro into what we
>> are
>> >>> >> trying to accomplish with Streams at Apachecon [1].  At the
>> conference
>> >>> >> there was a fair bit of interest and what seemed to me like
>> potential
>> >>> >> alignment; however, there are some serious issues to discuss and
>> work
>> >>> >> to resolution.  The issues I heard are as follows:
>> >>> >>
>> >>> >> 1) Everyone is interested, few are engaged.
>> >>> >>     There are quite a few people who could use this software, but
>> >>> >> aside from Jason, we really haven't seen much activity in
>> development
>> >>> >> or architecture.
>> >>> >>
>> >>> >
>> >>> > I'm one of the interested but not engaged yet. But I am interested in
>> >>> > engaging if I can make it fit for our project.
>> >>> >
>> >>> >>
>> >>> >> 2) The current code is not easy to engage with
>> >>> >>     We need to lay out tutorials on how to get started and discuss
>> >>> >> general architecture and package structure
>> >>> >>
>> >>> >> 3) We need processing pipelines for incoming and outgoing activities
>> >>> >>     How these pipelines are created, managed (workflow?),
>> parallelized
>> >>> >> and augmented by custom components needs to be discussed
>> >>> >>
>> >>> >
>> >>> > We do a lot of work with dynamically creating camel routes which
>> could be
>> >>> > used here potentially.
>> >>> >
>> >>> >>
>> >>> >> 4) We need to support some kind of persistence
>> >>> >>    Most people want to do ad-hoc querying rather than direct
>> >>> >> subscription notification (though both should be supported IMO), but
>> >>> >> we need persistence, indexes & a whole lot of thought to make this a
>> >>> >> reality
>> >>> >>
>> >>> >> 5) Dependencies need help
>> >>> >>    Someone brought up that all of our dependencies are versions old
>> >>> >> and need to be upgraded
>> >>> >>
>> >>> >
>> >>> > I think I might have made that comment. I believe this is because the
>> >>> > current deployment is tied to ServiceMix which isn't updated very
>> often.
>> >>> > It's picking up the pace again but ServiceMix is a big beast with
>> lots of
>> >>> > dependencies so it's slow to catch up with all the latest libraries.
>> >>> > ServiceMix 4.5 was just released which is using newer libraries which
>> >>> will
>> >>> > help.
>> >>> >
>> >>> > ** I just saw Jason's follow-up where he seemed to be suggesting
>> getting
>> >>> > rid of OSGI support, so if that is the case the stuff below isn't
>> >>> important
>> >>> > **
>> >>> >
>> >>> > 6) Code structure to support various deployments
>> >>> >     Right now I understand that there is a desire to deploy either
>> inside
>> >>> > an OSGI container or standalone as a WAR file in any servlet
>> container.
>> >>> > That is a great goal but I'm not sure the code is structured in the
>> best
>> >>> > way to take advantage of both scenarios to it's fullest. I have a
>> couple
>> >>> > thoughts of some simple changes that would allow the project to be
>> >>> deployed
>> >>> > in both places and still be able to take advantage of OSGI features
>> in
>> >>> that
>> >>> > environment.
>> >>> >  - Split the API from the implementation in separate bundles. This
>> can be
>> >>> > done two ways, either each capability has two bundles (-api, -impl)
>> or
>> >>> you
>> >>> > have one API bundle that includes all the API code for the whole
>> project.
>> >>> > This decouples the implementation from the API so the implementation
>> can
>> >>> be
>> >>> > updated without affecting all the bundles that use it. The downside
>> of
>> >>> not
>> >>> > doing it is that every time a bundle with an API class in it is
>> updated
>> >>> you
>> >>> > have to reload all the bundles that use that class definition.
>> >>> >  - Try to make the API/Implementations pure POJOs. There shouldn't
>> be any
>> >>> > common reason why you need to depend directly on OSGI classes if you
>> use
>> >>> > Blueprint for wiring (replace spring-osgi with blueprint). Having
>> pure
>> >>> > POJOs, with no OSGI or Spring dependencies, allows implementors to
>> wire
>> >>> > things up using the technology they see fit. (Might want to consider
>> a
>> >>> > profile here as well so that spring context file can be stripped.
>> >>> Otherwise
>> >>> > you wind up with multiple instances of the same beans being created
>> in
>> >>> OSGI)
>> >>> >  - For WAR files (if needed) there would probably need to be a
>> profile
>> >>> > setup to build either OSGI or non-OSGI. In the OSGI mode it wouldn't
>> >>> embed
>> >>> > all the libraries inside the WEB-INF/lib folder, instead it would use
>> >>> OSGI
>> >>> > imports to get those dependencies. It can also lookup services to use
>> >>> > instead of creating new versions of the beans.
>> >>> >  - For configuration parameters, in the Blueprint file use
>> configuration
>> >>> > properties. You can provide the default in the blueprint file but it
>> will
>> >>> > publish that item to the OSGI Config Admin service so it can be
>> updated
>> >>> > with either config files or directly through the API/web console.
>> >>>
>> >>> These suggestions seem very reasonable to me.  Any chance we can
>> >>> convince you to implement them? :)
>> >>>
>> >>
>> >> If the desire is to support both WAR and native OSGI deployments then
>> I'd
>> >> be glad to take a stab at it.
>> >
>> > IMHO, it would be nice to have both; but, only if we can clearly and
>> > cleanly define how to build for each case.  Your proposed enhancements
>> > appear to me to move further down that path.  Any one else have
>> > thoughts?
>> >
>> >>
>> >>>
>> >>> >
>> >>> > On a different note, there seems to be a lot of
>> NOTICES/DISCLAIMERS... in
>> >>> > the project. Are those required in every module? I believe those can
>> be
>> >>> > removed from any module that is a <package>pom</package> type since
>> that
>> >>> > package structure doesn't generate a jar or use the src folder
>> anyway.
>> >>>
>> >>> We should be able to pull these in during the build process so that
>> >>> they show up in the zip that gets released.
>> >>>
>> >>> >
>> >>> >>
>> >>> >> Anyone have any thoughts on these issues and how we as a community
>> can
>> >>> >> move past them?  Did anyone else have other issues to add to the
>> list?
>> >>> >>
>> >>> >> -Matt
>> >>> >>
>> >>> >> [1]:
>> >>> >>
>> >>>
>> http://archive.apachecon.com/na2013/presentations/26-Tuesday/Tapping_The_Stream/11:15-Apache%20Streams:%20Enterprise%20Social%20Integration.pdf
>> >>> >>
>> >>>
>>