qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Stitcher <astitc...@redhat.com>
Subject Re: custom store plugin for qpidd
Date Fri, 12 Aug 2016 17:31:43 GMT
I should state here that I've not actually ever implemented a store for
qpidd - those that have shold be at least lurking here and can identify
themselves!

But I think I some architectural understanding (well recollection
really!) of how they fit.

The first point is that there are actually 2 "stores" in qpidd and they
are not well separated:

1. There is a store for broker configuration - in the linear/legacy
stores this is a completely separate sleepycat db. I'm not sure how the
windows stores separate.

2. A store for durable messages.

I suspect you've already noticed this if you've got as far as
implementing a store.

On Thu, 2016-08-11 at 17:43 -0400, Matt Broadstone wrote:
> On Thu, Aug 11, 2016 at 10:42 AM, Matt Broadstone <mbroadst@gmail.com
> >
> wrote:
> 
> > 
> > Hey all,
> > 
> > I've been working for the past week on store plugin for RethinkDB
> > for
> > qpidd, and while I have made some pretty good progress I have a few
> > outstanding questions I was hoping someone here might have insight
> > into.
> > Here goes:

Are you planning to open source this plugin? It would be great to have
another store that worked on Unix, but wasn't tied to Linux only APIs.

> > 
> > There seem to be some notable differences between the
> > linearstore/legacystore and the MSSQL plugins.

As you say they conform to different interfaces.

As I understand it that is because the linear/legacy stores were
written in a way that assumes that only one store can be module loaded.
The legacy store was the first store written for the current store
interface, and it was simplest to just assume that only one store could
be loaded

Whereas the Windows stores (I believe that only uses MSSQL and the
other uses a different native journalling API) can both be loaded, but
opnly one can be active. These were written afterwards and I'm guessing
that because there were two written at the sam e time an indiection
layer was also written to switch between them.

It would make sense to switch the linearstore to use the layered API so
that multiple stores could be loaded and switched, but there has been
no interest in doing that work, largely because there has been no other
Linux store.

> >  If I'm not mistaken (and
> > please correct me if I'm wrong here), the MSSQL actually seems to
> > be more
> > of a reference implementation of the plugin api than the first two,
> > in that
> > the builtin stores are `qpid::broker::MessageStore`s while the
> > latter is a
> > `qpid::store::StorageProvider`. Which one should I be targeting
> > here?

If I remember correctly StorageProvider is the interface which allows
multiple stores to be loaded at once (although only one can be active
at any one time).

Architecturally I think StorageProvider sits on top of MessageStore.

> > ... {Details that I know nothing about)

> > Finally I just have some more general questions:
> >  - are there any resources available for implementors that I might
> > have
> > missed? Perhaps design documents, or some treasure trove of
> > communications

Sadly I'm pretty sure there is nothing available. The legacystore used
to be maintained as a separate plugin in a different repo, if you can
find that there may be something there.

> > ...

> Just another quick observation as I was cleaning up my cmake rules.
> Currently the "store" library is built as a module which I believe is
> incorrect but has probably worked so far since the only consumer has
> been a
> windows build.

Is the "store" plugin the one that allows switching between different
stores? If so then I agree it should be built directly into qpidd.

However this can only happen if the linear/legacy stores are modified
to use the API it exposes, as otherwise there would be two store
plugins vying for the exclusive right to be a store using that plugin
API.

>  Naturally, the store library should be a shared library that
> is directly linked into the storage providers that depend on it (you
> wouldn't `--load-module store --load-module mssql_store` just
> `--load-module mssql_store`). It seems on windows that module's are
> defined
> the same as shared libraries, so this probably worked just fine, but
> on
> linux it's a no-go and so linux store modules need to compile the
> parts of
> store that they need.

To be clear, on Linux the "store" module isn't used or needed because
the current Linux stores don't use it!

I hope this random spew of thoughts has been helpful or at least
amusing.

Andrew



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


Mime
View raw message