qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Broadstone <mbroa...@gmail.com>
Subject Re: custom store plugin for qpidd
Date Thu, 11 Aug 2016 21:43:25 GMT
On Thu, Aug 11, 2016 at 10:42 AM, Matt Broadstone <mbroadst@gmail.com>
wrote:

> Hey all,
>
> I've been working for the past week on store plugin for RethinkDB for
> qpidd, and while I have made some pretty good progress I have a few
> outstanding questions I was hoping someone here might have insight into.
> Here goes:
>
> There seem to be some notable differences between the
> linearstore/legacystore and the MSSQL plugins. If I'm not mistaken (and
> please correct me if I'm wrong here), the MSSQL actually seems to be more
> of a reference implementation of the plugin api than the first two, in that
> the builtin stores are `qpid::broker::MessageStore`s while the latter is a
> `qpid::store::StorageProvider`. Which one should I be targeting here?
>
> In terms of the actual implementation differences, a few things stuck out
> during implementation:
>
>   - the linearstore and legacy store actually store the message header and
> contents during an enqueue, while the MSSQL driver stores a preamble
> + qpid::framing::Buffer of the encoded `PersistableMessage`. I followed the
> latter, and have discovered it seems to actually store the whole message
> just fine surprisingly.
>
>   - the linearstore does some extra work when recovering messages,
> specifically calling `setRedelivered()` and `computeExpiration()` for the
> recovered message. The MSSQL plugin does not do this.
>
>   - the following methods are implemented in the MSSQL driver, and are
> implemented as methods that throw in the linearstore and legacystore
> plugins:
>
> void destroy(const PersistableConfig& config);
> void stage(const boost::intrusive_ptr<PersistableMessage>& msg);
> void destroy(PersistableMessage& /*msg*/);
> void appendContent(const boost::intrusive_ptr<const PersistableMessage>&
> msg, const std::string& data);
> void RethinkDBProvider::loadContent(const qpid::broker::PersistableQueue&
> queue, const boost::intrusive_ptr<const PersistableMessage>& msg,
> std::string& data, uint64_t offset, uint32_t /*length*/);
>
>   should they be? what's up with that.
>
>
> Finally I just have some more general questions:
>  - are there any resources available for implementors that I might have
> missed? Perhaps design documents, or some treasure trove of communications
> during the design process. It seems like there are some undocumented
> guarantees as evidenced by the divergent implementations mentioned above.
>
>  - as modification of the `run_windows_store_tests` script the accepted
> method of validating my implementation with the current state of the art?
>
>
> Thanks!
>
> Matt
>
>
>
Just another quick observation as I was cleaning up my cmake rules.
Currently the "store" library is built as a module which I believe is
incorrect but has probably worked so far since the only consumer has been a
windows build. Naturally, the store library should be a shared library that
is directly linked into the storage providers that depend on it (you
wouldn't `--load-module store --load-module mssql_store` just
`--load-module mssql_store`). It seems on windows that module's are defined
the same as shared libraries, so this probably worked just fine, but on
linux it's a no-go and so linux store modules need to compile the parts of
store that they need.

Matt

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message