brooklyn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ahgittin <...@git.apache.org>
Subject [GitHub] brooklyn-server pull request #746: [WIP] Auto wrap YAML BOMs as catalog bund...
Date Wed, 28 Jun 2017 11:31:34 GMT
GitHub user ahgittin opened a pull request:

    https://github.com/apache/brooklyn-server/pull/746

    [WIP] Auto wrap YAML BOMs as catalog bundles

    Wraps pure-YAML BOMs uploaded to catalog in a bundle, creating a name and version if needed.
    
    This allows us to see all items added as a unit.  This will allow us to prefer loading
co-defined items (though that's not fully done, still relies on preferring search path when
doing item lookup), and to edit or remove that item as a set (crucially we now keep the BOM,
previously we discarded it!).
    
    Docs changed to reflect this and explain differences in another PR.
    
    The basic wrapping was pretty simple but a surprising number of subtleties emerged which
this fixes:
    
    * java scanning doesn't work in OSGi; docs and errors make this clear
    * impacts replacement semantics (now it does a hash on the bundle to make most things
the same, though a re-post of a non-SNAPSHOT BOM that doesn't declare a bundle will now fail
because the random bundle name is stored on the item as containingBundle and is different
on a re-post; and needed extra work to remove leaked anonymous bundles)
    * had to change the way we set up the libraries / search path, to exclude this "bundle"
(mainly so tests are unchanged, but also improves efficiency for simple BOMs; also the code
is cleaner now)
    
    And there is one thing that emerged that I *haven't* fixed:  on rebind it became vital
to handle the situation where dependent bundles might not be loaded before the depender bundle;
see failing test`testLongReferenceSequence`.  Not entirely sure how to do this /cc @neykov
@geomacy @aledsage.
    
    Depends on #743.
    


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ahgittin/brooklyn-server osgi-auto-wrap-yaml

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/brooklyn-server/pull/746.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #746
    
----
commit 36ffdeeca0ac5290727b3bf5d2a7267664e9638a
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-16T08:49:44Z

    remove REST reference to removed method

commit 3860c3498d63a9a9d880480363c7947599d40a7c
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-16T10:53:33Z

    new utilities for registered type (catalog item id) naming syntax

commit 607fc853aa395915e15d527405b7a7a8a6e97b48
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-16T12:24:28Z

    move many things to use RegisteredTypeNaming rather than hokey `looksLikeVersionedId`

commit eb08f8ec0d65b59bb2c4201a98632aa070ddac97
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-16T12:38:45Z

    notes on related methods for looking up catalog items flexibly

commit c9d03c93e35eb5417ca6168207f4b0ddc3de9e7c
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-16T12:46:20Z

    remove deprecated CAMP ServiceTypeResolver
    
    changed a while ago to EntitySpecResolver; removes references to catalog items and hokey
lookup

commit db77f4e14d07fe9e23fb464fb23302afc1fba2f0
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-20T08:37:29Z

    fix tests that relied on removed deprecated behaviour

commit dc4942a345dc2fbd5125a34131addb632e634fe8
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-20T08:51:42Z

    Merge branch 'master' into version-id

commit 5b2cc48685ee606207257e74fd8a61f13f876a00
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-20T15:48:25Z

    prefer versions which comply with recommendations, heuristics elsewhere
    
    expand tests. some obscure ordering items are different now.

commit cc703928e1a2bd9c97795acab9a747a3733eb4a2
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-20T15:49:57Z

    tidy version comparator, removing old code

commit 45cb10a57af9e4aba0e7d624f1af6bfb25e5cee2
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-21T16:04:04Z

    add lots of text explaining BrooklynVersionSyntax, update comparator, tests, and usages

commit ed98e27e934ddf54d6c1c80b4a61afaab1225e47
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-21T17:26:43Z

    add routines to convert brooklyn/osgi versions
    
    and bundle finder supports either, and does the conversion for you

commit b46188e60e62247f8ffbe09c11acf37aaee69fad
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-21T17:33:06Z

    This closes #737

commit 5a7b64a751abdfbd3d75f20232a2c2a4dbb28feb
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-21T17:33:14Z

    Merge branch 'version-new-rules' into version-new-rules-prep

commit c5c5afc385341620f199737ddae3d2efaeb3a0a4
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-22T13:11:22Z

    This closes #737

commit d8c03c82a14f2d82c4280067322604bc0982a976
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-22T13:11:38Z

    This closes #740

commit 89fb13955eef518f53e3c3330a20191c89a3cf2e
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-22T12:17:32Z

    record underlying supplied version in VersionedName
    
    support for persistence, and test

commit 701a1d7fb70203fd6a0503c369b16674972344ac
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-22T12:18:28Z

    update OsgiBundleWithUrl to act like versioned name, and uses of VersionedName
    
    OSGi versions are taken at the very last moment, allowing us to preserve brooklyn-recommended
syntax throughout nearly all our code; and lookups/matches in OSGi space compare the OSGi-target

commit 97b50e4c304f0886f96a7cd503f1d634d1fbd9c4
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-26T11:49:38Z

    add routine to create new temp bundle

commit d7975d1414269f0f14617d3987170eba729c9c7d
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-23T09:53:51Z

    implicitly wrap yaml in bundles when uploading to catalog
    
    creates a new bundle when given yaml in catalog; bundle name and version now recommended
in the BOM. scan-java option tweaked but
    in a way that is consistent with the past and sensible in the new world. auto-wrapped
bundles are identified with a header so we can simplify their handling in some cases (scanning,
search paths).
    
    some things clumsy and need fixed:
    
    * uploading a different BOM (or bundle) at the same version says "ignoring because identical";
osgi identicality check should look
     at contents
    * auto-wrapped bundles are added to the library search path (no need for this as the bundles
are empty)
    * failed installations keep the bundle installed, blocking subsequent installs; probably
don't install unless forced?
    * warn if different version declared in BOM
    
    some things clumsy but we could live with:
    
    * containing bundles are added as libraries by munging the yaml; now we have a record
to that so can do a bit better than munge yaml
    * if bundle has no name, a random one is chosen (probably deprecated this behaviour);
if you re-submit we don't know it's the same bundle so we don't detect it's a bundle update;
this means:
      * uploading the exact same non-snapshot BOM twice will fail the second time saying the
items are different (because containing bundle is different)
      * uploading an updated item in a snapshot BOM will correctly replace, but the old bundle
will still be around (just masked; though we can fix that)

commit 2165778a354fa4221eec0e8ba03a96ef5bef92a3
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-27T13:23:34Z

    fix bundles set on scanning, and aborted work to scan osgi bundles
    
    scanning for OSGi can be made to work in some cases if we have the jar,
    but it won't work after rebind, or even in some cases if persisting (race deleting the
JAR),
    plus there are likely classloading issues;
    
    note OSGi deliberately doesn't give us the JAR.
    
    java scanning is discouraged by OSGi; it wants us to explicitly list things e.g. in the
catalog.bom,
    which is fine.

commit e7dc875af19138261f95994c5da08a2187ee64c7
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-27T14:38:15Z

    Add checksum to managed bundle so we can tell if it changes and fail accordingly
    
    This permits us to allow the same BOM YAML to be added multiple times (whether or not
bundles are declared), except where it's a non-snapshot BOM that doesn't declare a bundle,
because the containing bundle will be different there.
    This also fixes a bug where containingBundle wasn't persisted; not a big problem as we
re-read bundle items, but cleaner to have it working.

commit f16bda9310469ec35f4de8f3a084746abfe08a44
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-27T15:28:33Z

    support force when wrapping a BOM
    
    new batch of tests around catalog versioning with OSGi

commit 8385f133273116ec5df628325a7d71ad29d4dcb8
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-28T08:12:19Z

    repair tests where semantics changed, new tests
    
    including explicit test for new load order problems:
    * during rebind osgi bundles can have BOM loaded in any order, but they expect their deps
to be available
    
    two remaining problems:
    * wrapper library is added (changing assertions about numbers)
    * added containing library changes YAML (changing assertions about comments)

commit e32d74d15359152c9b1900db2ee0a7958b15c29b
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-28T08:28:13Z

    comments on BOM scanning

commit dc43f03fe2fa269e49a7865eafb491430f4e7e32
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-28T08:46:13Z

    add library bundles directly, without munging yaml, and only if not wrapper

commit 480a0c76ae434a7d05455ec0de63e33b5c05aebc
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-28T09:43:17Z

    Delete wrapper bundles who have no types installed.
    
    There is no point to them (they aren't OSGi).
    Fixes bug where uploading the same BOM multiple times, could cause bundle leakage.
    
    And tidy logging, more tests.

commit 21e93d27bed7efd3d8f115c8fc2234b39360ee3a
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-28T10:28:24Z

    restore test bundle counts, and resolve library URLs on install

commit 0236dbcd5809827c0f2548561c1ddfa5cc26d7a3
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-28T10:45:28Z

    address comments on PR in #737

commit 14aa7486ffbc07f57acf67836e5466205cb3b524
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-28T10:52:35Z

    Merge branch 'version-id' into version-new-rules-prep

commit ee853f5806f3d2d89f2c76521ec236695f61f80d
Author: Alex Heneveld <alex.heneveld@cloudsoftcorp.com>
Date:   2017-06-28T11:24:36Z

    tidy esp javadoc and regexes, as per PR comments

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message