river-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Greg Trasuk <tras...@stratuscom.com>
Subject Re: [Dicuss] Build Structure
Date Wed, 12 Feb 2014 22:16:15 GMT

Hi Dennis:

Some discussion inline…

Greg Trasuk

On Feb 12, 2014, at 4:16 PM, Dennis Reedy <dennis.reedy@gmail.com> wrote:

> Greg,
> I think how the River project is structured, built and tested becomes a recipe for how
developers that use River can begin to create their own projects. So IMO its something to
care about. As you point out, the current approach is based on decisions made over a decade
ago and there are reasons to change (as have been pointed out). 

Is the Tomcat source code a recipe for how to structure web applications?  All I’m saying
is that the purpose of River’s build system is to build River, and nothing more.  So let’s
think clearly about what we gain by restructuring it.  The biggest advantage I see is making
it easier to maintain and update the pieces separately (e.g. update Outrigger to use an open-source
data store instead of SnapLogStore, or create a clustered JavaSpace).  Eventually I think
that leads to multiple deliverables with different release cadences, but we certainly don’t
need to start out that way.

In any case, I think the recipe for how to develop applications has to be shown with good
examples, not River’s build system.  For application developers, the mechanism to build
the artifacts that go into Central isn’t important.  As they say, “People who like laws
and sausages probably shouldn’t watch either one being made”.

> If we choose to adopt conventions that produce a single jar per module,

Apart from my griping about having a lot of IDE projects, I think a single jar per module
is the way to go.  That will be much more clear about why the modules exist, and we’ll avoid
any possibility of having the same class in different jar files (which we currently do have).

> and that each module represent a basic architectural element of a distributed  service,
then we can begin to break out and organize the project. What I had started to do was to break
out River as follows:
> apache-river/
>    river-core/
>        jsk-dl/ 
>        jsk-lib/
>        jsk-platform/
>        jsk-resources/
>        serviceui/

Don’t forget jsk-policy.jar

Also, does anyone remember the rationale as to what goes into jsk-lib as opposed to jsk-patform?
 Right now there’s some overlap between them.  I thought the idea was to just have jsk-platform.jar.

>    river-services/ [1]
>        reggie/
>            reggie-proxy/
>            reggie-impl/
>        outrigger/	
>            outrigger-proxy/
>            outrigger-impl/
>            outrigger-snaplogstore/
>        mahalo/	
>            mahalo-proxy/
>            mahalo-impl/
>        mercury/	
>            mercury-proxy/
>            mercury-impl/			

Minor thing - could we keep it as “reggie-dl”, etc, rather than “reggie-proxy”?  To
me, *-dl expresses the intention a little better, since it’s all the classes that might
need to be downloaded to the client, which might include other support in addition to the
proxy.  Either naming will work, but it would be good to be consistent.  In building modules
for a container, it’s kind of nice to be able to pickup on the name of the jar to indicate
if it needs to go in the codebase annotation.

>    river-start/
>    river-tools/
>        browser/
>        classserver/

I’d almost put class server into the starter framework.  I don’t think it’s useful outside
the service-starter.

The browser is actually (in the current structure anyway) in the ‘examples’ directory.
 If you’re interested, I’ve already isolated it in the container build.  See https://git-wip-us.apache.org/repos/asf?p=river-container.git;a=tree;f=browser;h=cdb48103dced2273fc62ab3857676afec3861ef2;hb=HEAD
and https://git-wip-us.apache.org/repos/asf?p=river-container.git;a=tree;f=browser-module;h=1a5897a11e1419055eefdfe6d439bd2d2f4d1cb0;hb=HEAD.

>    river-integration-tests

I agree that the integration tests should probably stay as a separate module.  As time goes
on it would be good to have unit tests in the other modules, too.

> Under each module (example reggie-proxy), we would have:
> src/
>  main/
>    java/
>    groovy/
>    (other languages...)
>    resources/
>  test/
>    java/
>    groovy/
>    (other languages...)
>    resources/
> I think the most difficult and time consuming effort would be to try and merge in the
current unit tests into each module. AFAIK, mots of the tests are integration tests, so putting
them in their own module makes sense.
> We still have one deliverable, all we need to do is "assemble" the project into a distribution.
This is a fairly simple thing to do with Maven (I'm sure it's not that difficult to do with
Gradle either).
> Regards
> Dennis
> 1. In most cases an api module is not required for River core services since the API
is defined is jsk-dl

That’s probably open to argument.  For example, is JavaSpace a “core” API?  It always
seemed to me like it landed inside Jini because it needed a home (I recall that at one time
it was a separate project), but it seems more like an application API.  I’m not saying it
_should_ or _shouldn’t_ be split out, but it’s something to think about.  Things like
Registrar, however, seem to be pretty “core”.

Overall, a good start on the modules.  We’ll need to settle on a tool (maven, Gradle, Ant,
etc).  I’m kind of with Simon on this - Maven is pretty standard, and I’d probably vote
for it in the absence of strong arguments for a different build tool. Although I did notice
as I was doing the container work that it sometimes does impose limits on the actual application
(for example, I couldn’t for the life of me figure a clean way to get it to generate a module
with a “.ssar” extension without writing a plugin, which I wasn’t at the time aware
how to do, so I changed the deployment module to “.jar”). 

I’m assuming we’d wait to modularize until Peter declares ‘qa_refactor’ done, and
merges it back into the trunk?

> On Feb 12, 2014, at 1034AM, Greg Trasuk <trasukg@stratuscom.com> wrote:
>> Hi all:
>> I’d just like to sound a quiet note of caution here.  
>> Everyone seems to be in agreement that the current build structure is problematic.
 After all, it was fundamentally designed around the ‘make’ tool 15 years ago.
>> However… Altering the build structure and switching to a new tool (Maven, Gradle,
Ant/Ivy, etc) will be a _lot_ of work, and we’ll be living with it for a while.  Let’s
not jump to conclusions without a good discussion.
>> So let’s start with some reflection…
>> It’s worth asking “who cares about the build system?”  Let’s be clear about
something:  the vast majority of developers should be using the jars out of Maven Central,
or at most running the River build once to build from source.  It’s only us, on the River
project, who are using the build system.  That being the case, perhaps we ought to talk about
what we intend to do, before we make any decisions or expend a lot of effort.
>> Are we thinking in terms of runtime containers or execution environments?  The core
build doesn’t matter then.  For example, the container work in progress (https://git-wip-us.apache.org/repos/asf/river-container.git)
builds with Maven, but uses the River dependencies published in Maven Central.
>> Are we planning separate modules for Reggie, Mahalo and Outrigger?  Those new modules
will need a build/integration test system, and we should discuss that.  Worth mentioning that
so long as the jars go into Maven Central or some other dependency management system, the
modules don’t actually need to have the same build tool (although consistency is probably
>> What else are we planning?  Chances are, it should go into a different module.
>> Which brings me to another topic - how should we manage multiple deliverables?  Could
I suggest the following “deliverable sets”, which would each have different releases?
>> - Jini Core - Basically what’s currently in jsk-platform.jar
>> - Registrar Service Module
>> - Transaction Service Module
>> - JavaSpaces Service Module
>> - note - I’m not sure if Mercury, fiddler, etc are used all that much, but if so,
they could have their own modules
>> - Jini Application Server (river-container packaged with the infrastructure services)
>> - other - Rio?  Anything else?
>> The actual set of modules is open to discussion of course, but I’m thinking we
need to support more than one “deliverable”.  Not quite “sub projects”, since the
governance doesn’t need to be separate, but I think separate source repositories and releases.
>> In any case, my big suggestion is let’s not take these decisions lightly.  There’s
no hurry.
>> Cheers,
>> Greg.
>> On Feb 12, 2014, at 5:31 AM, Simon IJskes - QCG <simon@qcg.nl> wrote:
>>> On 12-02-14 11:28, Simon IJskes - QCG wrote:
>>>> Ok,
>>>> what i have seen is:
>>>> - maven / gradle / ivy
>>>> - convention (submodules structure)
>>>> - 1 river or submodules (reedy-trasuk)
>>>> Do we need to discuss this further, or do we need a vote on the sub-items.
>>>> Gr. Simon
>>> in order of pref:
>>> - maven
>>> - gradle
>>> - ivy
>>> conventions:
>>> +1
>>> submodules:
>>> +0
>>> Gr. Simon
>>> -- 
>>> QCG, Software voor het MKB, 071-5890970, http://www.qcg.nl
>>> Quality Consultancy Group b.v., Leiderdorp, Kvk Den Haag: 28088397

View raw message