ace-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcel Offermans <marcel.offerm...@luminis.nl>
Subject Re: Where does Ace server store everything?
Date Tue, 06 Aug 2013 20:58:00 GMT
Hello Matthew,

On Aug 6, 2013, at 22:15 PM, Matthew Bishop <matt@thebishops.org> wrote:

> Thanks for the quick reply. What I am really looking for is a declarative
> way to express every bundle and configuration in a distribution, but in a
> way that can be checked into source control and diffed for changes over
> time.

Let me try and explain how the client API and the repositories work. I will use the Web UI
as an example, because that is the easiest to visualize.

Conceptually, ACE consists of a server part, where the repositories are stored, and a client
part, that checks out, manipulates and stores new versions of repositories. So, whenever you
open the Web UI, upload new versions of bundles, change features and distributions and assign
them to targets, you're manipulating these repositories and when you hit store, they are committed
to the server.

On the server, we keep all versions of stores over time, like in a versioning control system.
Its API is like this (I removed comments and exceptions for brevity):

public interface Repository {
    public SortedRangeSet getRange();
    public boolean commit(InputStream data, long fromVersion);
    public InputStream checkout(long version);
}

So in a way, the Web UI lets you "declare" your distributions already, and ik keeps a full
history of all changes you made. In fact, you could re-implement the Repository interface
above to store that data in some revision control system.

In ACE, there are actually three repositories that store the data:

1) A store repository, that holds the artifacts, features and distributions (and their associations).
2) A target repository, that holds the mapping between the distributions and the targets.
3) A deployment repository, which is derived from the store and target repositories and contains
all the versions of the software for each target, and for each version all the artifacts that
are part of that version.

When a client checks out a repository (let's take the store as an example) it gets the latest
version from the server. That's a gzipped XML definition, that is mapped (by XStream) to an
in-memory Java object model. That model is what you manipulate through the client API. Finally,
you store that back, it gets transformed to XML, gzipped, and sent back to the server.

Diffing such versions is certainly a possibility (although it would be some work, simply doing
a text based diff on the XML probably does not give you the information you are looking for).

On the other hand, we have also written some code in the past to define distributions based
on (in our case) bndrun files (the format used by Bndtools to define a run configuration).
That involved writing some glue to convert that format to a list of artifacts, linked to a
feature and distribution.

> Ideally this expression (files) could be pushed to a target instance which
> would trigger the fetching of missing resources and the configuration of
> the instance to match.
> 
> It looks like Ace can manage the latter part of this just fine, but the
> first part (declarative distribution) is what is missing. We don't want
> manage server configurations by hand any more than we want to build WAR
> files by hand.

I understand.

To give you another example, we recently automated our continuous build system so for each
succesful build it takes the artifacts that come out of that build and (through a GoGo Shell
script) upload them to a repository and then create all the metadata in ACE. The end result
is that we can automatically deploy such code directly to our QA and demo environments, completely
automated.

Greetings, Marcel

> On Tue, Aug 6, 2013 at 1:03 PM, Marcel Offermans <
> marcel.offermans@luminis.nl> wrote:
> 
>> Hello Matthew,
>> 
>> On Aug 6, 2013, at 21:50 PM, Matthew Bishop <matt@thebishops.org> wrote:
>> 
>>> I just started looking into Ace and have set up several artifacts,
>> features
>>> and distributions using the web UI. I have added a target, then Stored
>>> everything.
>> 
>>> Next I want to see what ace stores on disk. The reason for this is I want
>>> to know how declarative it is, and if the saved data can be generated
>> from
>>> some way other than the API and the UI.
>> 
>>> So where are the definitions stored? I cannot locate them anywhere, and
>>> before I download the source and start spelunking I wanted to ask the
>> group.
>> 
>> The data is stored in the bundle cache. More specifically, in the bundle
>> cache of the repository implementation (org.apache.ace.repository.impl). It
>> is possible to locate this cache on disk, and you will find binary blobs
>> there. They are gzip'ed, and if you unpack them you will discover XML data.
>> 
>> In theory, you can edit that data directly, but I would strongly
>> discourage that as this is an internal format that we have and in the
>> future will change if necessary. Really your best bet is to manipulate this
>> data through one of the three clients we have:
>> 
>> - Web UI (you already stated you don't want to use that)
>> - REST client API (a set of REST endpoints)
>> - GoGo Shell client API (a set of commands for the built-in GoGo shell)
>> 
>> All of these build on a set of Java Client APIs.
>> 
>> If you want some kind of declarative format for specifying the data you
>> want to put in ACE, my suggestion would be to define such a format and then
>> write some code to take that format and put it in ACE using the Java client
>> API (probably taking one of the three existing clients as an example).
>> 
>> However, perhaps you should explain your use case a bit more before I
>> start giving the wrong advice. :)
>> 
>> Greetings, Marcel
>> 
>> 


Mime
View raw message