brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John McCabe <j...@johnmccabe.net>
Subject Re: Repository splitting script
Date Wed, 16 Dec 2015 09:55:24 GMT
@ahgittin going to pick this up again today, tomorrow sounds like it might
be a bit tight tho, will see how I get on this morning and update.

On Tue, 15 Dec 2015 at 18:08 Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
wrote:

>
> Hi All-
>
> I've been hacking at
> https://github.com/rdowner/brooklyn-repo-split/pull/5 to move the
> restructuring along.  I'm testing this now but I think we now have much
> of what we want working.  I'm testing it now.
>
> *@johnmccabe* is working on the big task remaining which is to make all
> the pom files work with tests in the right place. John can you work in a
> branch of incubator which is the result of step `1-*.sh` (in the branch
> of PR 5) ?
>
> And *@/all* when would be a good time to switch the incubator repo to
> the new structure?  Can I suggest Thursday, if that's enough time for
> John to make the new-structure poms work?
>
> It will mean that existing PR's can't be merged automatically -- but
> with a merge they should mostly do the right thing, modulo any new files
> which have to be merged, unless you've hit some of the poms or tests
> which were fixed?
>
> Best
> Alex
>
>
> On 14/12/2015 13:01, Alex Heneveld wrote:
> >
> > Nice work.  Answers in-line.
> >
> > One general point however:  I strongly suggest we do the restructuring
> > and maven tweaking in the incubator-brooklyn repo. This will mean the
> > cut-over to the new repos is only a matter of separating history for
> > subdirs.  This allows us to have confidence that everything is working
> > -- esp maven -- when we cut across, it splits the problem up into two
> > simpler problems, and it makes the cut-over transparent -- e.g. even
> > if there is WIP in someone's incubator, when they rebase on latest
> > incubator they will have the right structure to apply a diff patch to
> > the new repo-set filesystem.  (I've tested this and confirmed it works
> > for moved files; although it will leave out new files, so you'll have
> > to eyeball them if you apply this method.)
> >
> > This means we can open a PR with the restructuring quite soon.
> >
> > It would be great if we could do that this week, and then move to the
> > new brooklyn repos over the weekend.  And then maybe even get a 090
> > non-incubating out before Christmas!
> >
> > We've got some great new stuff to include, including the CLI and the
> > test framework, and hopefully Salt/Ansible support and OSGi and YAML
> > context highlighting.
> >
> >
> >> 1. multiple dependencies on brooklyn-software-base (CAMP/REST
> >> Server/Launcher etc) - I suspect this belongs in the brooklyn-server
> >> repo
> >> rather than brooklyn-libraries (I'd added it back into my test repo [1]
> > Agree.  Wrote last Wed suggesting this should be in brooklyn-server/
> > software/base  (as well as  software/winrm ) .
> >
> >> 2. qa module heavily dependant on brooklyn-software-* from the
> >> brooklyn-library repo - move to the  brooklyn-library repo or
> >> introduce a
> >> new repo? (I'd disabled it in [1])
> > I think move QA to brooklyn-library.
> >
> >>
> >> 3. tests in brooklyn-camp module depending on brooklyn-software-* from
> >> brooklyn-libraries repos  (EnrichersSlightlySimplerYamlTest,
> >> EntitiesYamlIntegrationTest, JavaWebAppsIntegrationTest,
> >> MapReferenceYamlTest, ObjectsYamlTest) - refactor (?) or move (I just
> >> commented these out for the moment)
> >
> > I think we should:
> >
> > * copy these tests to the appropriate  software-XXX bundle; I think
> > it's fine if they have test dependencies on the CAMP module
> >     * (or move to the QA bundle)
> >
> > * with answer to (1) above we can still use software-base
> >
> > * for others I think comment out the offending tests in the camp
> > module, with a TODO saying that we'd like to create variants of those
> > tests which do not depend on specific software blueprints (or if easy,
> > write them now)
> >
> >
> >> 4. unable to build brooklyn-server with -Dmaven.test.skip=true as there
> >> were dependencies on generated test jars (didn't dig into this,
> >> commented
> >> out problem tests - see #3)
> > should be fixed
> >
> >>
> >> 5. tests in brooklyn-ui depend on  brooklyn-rest-server,
> >> brooklyn-test-support, brooklyn-api, brooklyn-core etc (built
> >> with -Dmaven.test.skip=true for the moment)
> > i think it's fine for ui to have a *test* dependency on server.
> > ideally the ui project have two different test modes, one of which
> > uses JSON files (selenium/jasmine style -- there is code in here for
> > that), and another which uses a real REST server.
> >
> >
> >> 6. both brooklyn-launcher and karaf apache-brooklyn (Distro) currently
> >> depends on brooklyn-jsgui war from brooklyn-ui repo for build, both will
> >> build without it present but launching brooklyn throws
> >> 'java.io.IOException: brooklyn.war not found on classpath', supplying
> >> '--startupContinueOnWebErrors' allows start to proceed but the REST API
> >> isn't accessible. What is the desire here, to start without a UI by
> >> default
> >> - ie just the REST API? Start the UI automatically if detected on the
> >> classpath or require the user to explicitly request a UI be started?
> >
> > i think we move usage/cli ("brooklyn-cli") to be
> > brooklyn-server/server-cli-base/ "brooklyn-server-cli-base" and not
> > run a UI by default, but with the ability to be subclassed.
> >
> > and we create   brooklyn-dist/server-cli "brooklyn-server-cli" which
> > only specifies that it should use the brooklyn.war (for which it will
> > have visibility of course).
> >
> >> 7. do we want to create equivalents to brooklyn-all/brooklyn-parent for
> >> each repo (brooklyn-server-all, brooklyn-library-all etc)
> > i don't think there's a need for this
> >
> >
> >>
> >> 8. pulling it all together... any suggestions/ideas on the proposed
> >> brooklyn repo?
> >>
> > as suggested above, let's do it in a PR on incubator-brooklyn
> >
> >
> > Best
> > Alex
> >
> >
> > On 14/12/2015 12:40, John McCabe wrote:
> >> I've got server/library/ui/dist building but with quite a few
> >> caveats/problems.
> >>
> >> 1. multiple dependencies on brooklyn-software-base (CAMP/REST
> >> Server/Launcher etc) - I suspect this belongs in the brooklyn-server
> >> repo
> >> rather than brooklyn-libraries (I'd added it back into my test repo [1]
> >>
> >> 2. qa module heavily dependant on brooklyn-software-* from the
> >> brooklyn-library repo - move to the  brooklyn-library repo or
> >> introduce a
> >> new repo? (I'd disabled it in [1])
> >>
> >> 3. tests in brooklyn-camp module depending on brooklyn-software-* from
> >> brooklyn-libraries repos  (EnrichersSlightlySimplerYamlTest,
> >> EntitiesYamlIntegrationTest, JavaWebAppsIntegrationTest,
> >> MapReferenceYamlTest, ObjectsYamlTest) - refactor (?) or move (I just
> >> commented these out for the moment)
> >>
> >> 4. unable to build brooklyn-server with -Dmaven.test.skip=true as there
> >> were dependencies on generated test jars (didn't dig into this,
> >> commented
> >> out problem tests - see #3)
> >>
> >> 5. tests in brooklyn-ui depend on  brooklyn-rest-server,
> >> brooklyn-test-support, brooklyn-api, brooklyn-core etc (built
> >> with -Dmaven.test.skip=true for the moment)
> >>
> >> 6. both brooklyn-launcher and karaf apache-brooklyn (Distro) currently
> >> depends on brooklyn-jsgui war from brooklyn-ui repo for build, both will
> >> build without it present but launching brooklyn throws
> >> 'java.io.IOException: brooklyn.war not found on classpath', supplying
> >> '--startupContinueOnWebErrors' allows start to proceed but the REST API
> >> isn't accessible. What is the desire here, to start without a UI by
> >> default
> >> - ie just the REST API? Start the UI automatically if detected on the
> >> classpath or require the user to explicitly request a UI be started?
> >>
> >> 7. do we want to create equivalents to brooklyn-all/brooklyn-parent for
> >> each repo (brooklyn-server-all, brooklyn-library-all etc)
> >>
> >> 8. pulling it all together... any suggestions/ideas on the proposed
> >> brooklyn repo?
> >>
> >> I've used the build_wip branch on the following repos (generated
> >> using [2])
> >> to investigate (I set the version to 0.9.1-SNAPSHOT to avoid clashing
> >> with
> >> the current 0.9.0-SNAPSHOT artifacts):
> >>
> >> - https://github.com/johnmccabe/TEMP-brooklyn-server/commits/build_wip
> >> - https://github.com/johnmccabe/TEMP-brooklyn-library/commits/build_wip
> >> - https://github.com/johnmccabe/TEMP-brooklyn-ui/commits/build_wip
> >> - https://github.com/johnmccabe/TEMP-brooklyn-dist/commits/build_wip
> >>
> >> You can currently build a dist in the following order (TODO the poms
> >> need
> >> some serious tidying):
> >>
> >> *TEMP-brooklyn-ui*
> >> mvn clean install -Dmaven.test.skip=true
> >>
> >> *TEMP-brooklyn-server*
> >> mvn clean install
> >>
> >> *TEMP-brooklyn-library*
> >> mvn clean install
> >>
> >> *TEMP-brooklyn-dist*
> >> cd usage/all
> >> mvn clean install
> >> cd ../../usage/dist
> >> mvn clean install
> >>
> >>
> >> [1]
> https://github.com/johnmccabe/TEMP-brooklyn-server/commits/build_wip
> >> [2]
> >>
> >> On Wed, Dec 9, 2015 at 10:48 PM, Alex Heneveld <
> >> alex.heneveld@cloudsoftcorp.com> wrote:
> >>
> >>> i think leave it `brooklyn/software/winrm` in the old hierarchy, and
> >>> in the
> >>> new hierarchy `brooklyn-server/software/winrm` alongside
> >>> `brooklyn/software/base`
> >>>
> >>> `locations/*` is intended for implementations of `Location`
> >>>
> >>> --a
> >>>
> >>>
> >>> On 9 December 2015 at 22:36, John McCabe <john@johnmccabe.net> wrote:
> >>>
> >>>> Does locations/winrm sound reasonable? (rather than reverting back to
> >>>> /core)
> >>>>
> >>>> On Wed, Dec 9, 2015 at 5:19 PM, Aled Sage <aled.sage@gmail.com>
> wrote:
> >>>>
> >>>>> +1
> >>>>>
> >>>>> Let's move winrm to brooklyn-server.
> >>>>>
> >>>>>
> >>>>>
> >>>>> On 09/12/2015 16:56, Hadrian Zbarcea wrote:
> >>>>>
> >>>>>> That should be (or at least become) a soft dependency, not a
hard
> >>>>>> one.
> >>>>>> That aside, the winrm piece probably belongs in server.
> >>>>>>
> >>>>>> Hadrian
> >>>>>>
> >>>>>> On 12/09/2015 11:27 AM, John McCabe wrote:
> >>>>>>
> >>>>>>> Alex, all, (fyi Hadrian/Ciprian),
> >>>>>>>
> >>>>>>>
> >>>>>>> * fix pom files on result of `rearrange-incubator.sh` script
so it
> >>>> builds
> >>>>>>>>> (make this a diff / git cherry-pick we can just
apply once all
> >>>>>>>>> PRs
> >>>> are
> >>>>>>>>>> merged?)
> >>>>>>>>>>
> >>>>>>>>>> John also said to me that he would take a look
at this
> >>>>>>>>>> problem. He
> >>>> was
> >>>>>>>>> temporarily prevented from doing so by a bug in
the split script
> >>>> which
> >>>>>>>>> broke TEMP-brooklyn-server, but that dodgy repo
should now be
> >>> fixed.
> >>>>>>>>> John - any update?
> >>>>>>>>
> >>>>>>>>>> Just after having a chat with Richard about
this, and am in the
> >>>>>>> process of
> >>>>>>> working through it.
> >>>>>>>
> >>>>>>> Have hit one problem so far, a circular dependency between
> >>>>>>> brooklyn-server
> >>>>>>> and brooklyn-library from some of the OSGI-ification changes.
> >>>>>>>
> >>>>>>> The 'Brooklyn Jclouds Location Targets' module in brooklyn-server
> >>> has a
> >>>>>>> dependency on the 'brooklyn-software-winrm' artifact from
> >>>>>>> brooklyn-library,
> >>>>>>> but brooklyn-library itself depends on brooklyn-server.
> >>>>>>>
> >>>>>>> Any suggestions on what we should do with it? There's mention
of
> >>>>>>> additional
> >>>>>>> cleanup required later in the commit (see pull#957 [1] (JIRA
> >>>> BROOKLYN-182
> >>>>>>> [2]))
> >>>>>>>
> >>>>>>> /John
> >>>>>>>
> >>>>>>> [1] https://github.com/apache/incubator-brooklyn/pull/957
> >>>>>>> [2] https://issues.apache.org/jira/browse/BROOKLYN-182
> >>>>>>>
> >>>>>>>
> >
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message