Mailing-List: contact ace-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: ace-dev@incubator.apache.org
Received-SPF: softfail (nike.apache.org: transitioning domain of
 marcel.offermans@luminis.nl does not designate 194.109.24.38 as permitted
 sender)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1084)
Subject: Re: Versioned OBR proposal
From: Marcel Offermans <marcel.offermans@luminis.nl>
In-Reply-To: <31811.8476.19403-8645-10686737-1322694358@email.cz>
Date: Sun, 4 Dec 2011 23:37:53 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <B623E8F9-2D34-4F1F-869A-6365E7CFCA7A@luminis.nl>
References: <31811.8476.19403-8645-10686737-1322694358@email.cz>
To: <ace-dev@incubator.apache.org>

Hello Jan,

On Dec 1, 2011, at 0:05 AM, <janzeman@email.cz> <janzeman@email.cz> =
wrote:

> as a member of a research group in Pilsen, that strives to create=20
> a tool for semantic version of OSGi bundles =
(http://osgi.kiv.zcu.cz/obvs/), =20
> I have some questions to ask:
>=20
> I don't think OBR from projects ace-obr-servlet and ace-obr-storage=20
> is accessed via the Vaadin UI interface. Instead of it, as deep=20
> as I've dug, it looks like it uses the ArtifactRepository from=20
> ace-client-repository-* projects. Does this mean, the projects ace-obr=20=

> projects are obsolete? Or is the ace-obr-servlet really mapped to=20
> http://localhost:8080/obr only without the web interface? How can one=20=

> then manage this OBR, e.g. put the bundles there?

To start with your first question: no, the OBR bundles are not obsolete =
at all. To explain how and where they're used, I should perhaps first =
explain a bit more about the ACE server and which major subsystems it =
includes. In this context, there are three subsystems which are =
relevant:

1) the OBR bundles
2) the client bundles
3) the (other) server bundles

These three subsystems can actually be deployed on different nodes, even =
though by default we all ship them as part of the ACE server.

The OBR bundles, when deployed with an HTTP server, form a stand-alone =
OBR which publishes (as per OBR spec) the contents of its repository via =
the (automatically generated and updated) repository.xml and extends the =
normal spec a bit because we also implemented an HTTP POST call to add =
new artifacts to the OBR. For convenience, if you have physical access =
to the filesystem that OBR runs on, you can even directly copy artifacts =
into the "store" folder and they will be detected as well (even though =
we don't actively encourage that, the official way is via the POST =
method as that allows us to properly check the contents of the =
artifact).

The client bundles, together with the Vaadin web UI and an HTTP server, =
can also be deployed on a different node. Actually, the client bundles =
form an API that can be used in all kinds of clients. Before donating =
the codebase to Apache, for example, we had a Swing based "rich client" =
that could run on a different machine and that communicated with both =
the ACE server and the OBR. This is also why you see that artifacts =
"enter the system" via the client API. If you dig deeper, you will see =
that from the client, an artifact will be "uploaded" to the OBR as well, =
via a POST call.

Finally, the other server bundles form the heart of the ACE server and =
contain the major repositories that hold all historic metadata for the =
store repository (which keeps track of the relations between artifacts, =
features and distributions), the license repository (which keeps track =
of the relations between distributions and targets) and the deployment =
repository (which is really the sum of the first two, and maps targets =
to artifacts in a format that is suited for fast deployment). During =
deployment, when a management agent contacts the server and asks for a =
deployment package, this package is actually created on the fly by the =
server. During that process, it will fetch artifacts from the OBR (so =
that's another place where the OBR is used).

> Some time ago, our leader Premek Brada proposed adding our semantic=20
> versioning to the OBR of ACE. We would like to go on with this goal. =
Some=20
> changes were made and I'd like to present it to the you and ask for =
your=20
> opinion :
>=20
> 1. PUT
> Our vision is that by putting a new resource to the OBR, there=20
> would be a decision, whether it is or it isn't a bundle. If it is =
another file,=20
> store it only in the OBR. If it is a bundle, perform the versioning =
according=20
> the last version of the bundle in the OBR first, then store it. This =
should
> happen all the time and thus, we could have the consistent list of all =
the=20
> versions of the bundle, that were uploaded in time. Naturally, if =
there is no=20
> another version, no versioning is performed.=20

If you look at the definition of a REST API then the semantics for PUT =
are to always store (and replace) the exact resource that you specified, =
so I would not want to deviate from that unless we have a really good =
reason. We are however free to add a POST command that uses semantic =
versioning.

> The problem is, that only the method put(String fileName, InputStream=20=

> data) is present so far, the client decides the filename. If there is =
a file with=20
> this name in the OBR (which is the case by different bundle versions) =
it would=20
> be overwritten.

We are free to extend our API.

> To let the OBR API backward compatible and still take advance=20
> from the versioning, the client have to manually add the the version =
suffix to the=20
> filename, but it doesn't know, whether the version in the manifest is =
really=20
> correct, since no semantic versioning was performed yet.=20

Right now we assume the client has correctly done semantic versioning. =
Most of the time, he will have to do that by hand, but there are other =
tools surfacing that help (BndTools now allows this).

> We propose to add another method to the OBR servlet like=20
> dryRun(InputStream data), that perform the versioning on the uploaded =
bundle=20
> but doesn't store it to the OBR, only returns in the HTTP header the =
info, whether=20
> there is another version and if so, also the corrected version of the =
uploaded=20
> bundle. Then the client constructs the filename and calls the common =
put=20
> method.

Let me make a counter-proposal here to add a POST method that will do =
the upload, then modify the bundle (if there are existing versions in =
the OBR of this same bundle) and finally either redirect or return the =
proper URL of the versioned bundle.

> 2. GET
> Typically I like to get the latest version of the bundle, that is in =
the repository.
> We can use the present get(filename) method, but we have to know the =
filename.
> listBundles(String symbolicName) would do the work, possibly by =
inspecting
> the repository.xml.

The current OBR spec indeed assumes that you first GET the =
repository.xml, parse that and based on the information inside that =
file, make up your mind about what exact bundle (or artifact) you want.

On top of that, I could live with an optional query mechanism that =
allows you to do things like:

GET /obr/?bsn=3Dorg.apache.felix.cm - to return the highest version of a =
bundle with the specified symbolic name
GET /obr/?vsn=3Dorg.apache.felix.cm&v=3D[1.0,2.0) - same, but highest =
version within the given range
GET /obr/?vsn=3Dorg.apache.felix.cm&v=3D[1.0,2.0)&lowest - same, but =
lowest version within the given range

or even, some kind of query that will return a list or subset (in which =
case it might actually make sense to either return just URLs or a =
filtered repository.xml)

Greetings, Marcel=