Mailing-List: contact dev-help@zest.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@zest.apache.org
MIME-Version: 1.0
Sender: hedhman@gmail.com
In-Reply-To: <55DBA8F3.9090004@gmail.com>
References: <55DBA8F3.9090004@gmail.com>
From: Niclas Hedhman <niclas@hedhman.org>
Date: Tue, 25 Aug 2015 21:31:06 +0800
Message-ID: 
 <CADmm+KdjRhkcB2a7Z0fn4a3JxMx0-cNLAAonu1aUjjYiRidfew@mail.gmail.com>
Subject: Re: A few zest 3.0 thought?
To: dev <dev@zest.apache.org>
Content-Type: multipart/alternative; boundary=089e0149422030eae5051e22c007

--089e0149422030eae5051e22c007
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Yes, you are making sense, and aligns with my thoughts.

The challenge lies in the details, and why QI4j pre-1.0 got seriously
burned, by something we don't even have a name for (call it Visibility
migration). The UnitOfWork adjusts the visibility as the calls traverses
the modules and layers. So, although the UoW is created high up, it needs
to "adapt" to visibility for each composite/object asking for something.
The mechanism for this is both very delicate, not obvious, and hard to
reason around. It is also likely to be filled with bugs.

Most of the Persistence does the right thing, but in Indexing it works for
rdf-indexer, but IIRC it doesn't work in the sql-indexer. A lot of that is
probably related to broken separation of concerns, in that UoW should
"somehow" ensure that the SPI level stuff doesn't need to worry about it.

And it could be that there is missing a "ViewPoint" concept, i.e. the
location from where the Visibility is derived, and that the ViewPoint might
be even more central to the invocation stack than belonging inside the UoW,
since I assume UoW isn't necessarily required (OR IS IT??).

Hard to discuss this on mailing list, and a whiteboard and couple of days
of time would sure be more efficient.

I more than welcome you to drive this with some stylized examples, that
tries to take into consideration what happens during persistence, querying
and other future "services" in such a world.


Cheers
Niclas

On Tue, Aug 25, 2015 at 7:29 AM, Kent S=C3=B8lvsten <kent.soelvsten@gmail.c=
om>
wrote:

> Gang,
>
> A lot of stuff has been discussed during the summer - suggestions for an
> upcoming 3.0 release.
> A lot of it consists of new composite types and new ways of persistence
> - thus challenging the existing way of persisting stuff.
>
> A Zest application consists of a single application with a CONFIGURABLE
> number of layers
> Each layer has a CONFIGURABLE number of modules.
> Each module has a FIXED set of structural parts: The UOW and
> implementations of some factories, each factory handling the lifecycle
> of a single composite type.
> The UOW stuff implements a FIXED set of lifecycle interfaces, adding
> ways to query, fetch and manipulate entities (a single composite type).
>
> Below the fixed API a few service provider interfaces allow for several
> implementations, especially for storing and querying stuff.
>
> We have already stuff challenging that model, since the native queries
> (SOLR/SERQL if i recall correctly) are not a great fit.
> We have the spatial query stuff, which is great, but the existing model
> is not really prepared to add that.
>
> In the pipeline we have more stuff ....
>
> Message composites with a message store?
> Aggregates with an event store?
> And maybe some ORM stuff too?
>
> The "natural" way to add that would be by adding more factories to the
> UOW - and maybe some more query interfaces to the modules.
> Probably not a good idea to overload those already large interfaces -
> and a suggestion that has already (implicitly?) been rejected for the
> native query stuff.
>
> I think we simply *need* a way to extend the core with new stuff in a
> natural way.
>
> Could we use services for that?
>
> What would happen if we removed
> ObjectFactory/ValueBuilderFactory/TransientBuilderFactory from the
> module itself, and added those as services instead?
> Application code could no longer use the functionality from the module
> directly, but would have to use the ServiceFinder or @Structure injection=
s.
> A bonus would be adding visibility - allowing assemblers to enforce that
> some of the factories can only be used from inside the module.
>
> Not trivial to implement, but not that bad either, I think.
>
> Somewhere down the road, that might even suggest a way to add new
> composite types. Bootstrapping TransientComposites could in some way be
> regarded as syntactical sugar around configuring the
> TransientBuilderFactory structural service - and maybe the model
> definitions could even live inside that service instead of inside the
> module?
>
> But .....
>
> The hairy stuff is the persistent stuff living inside the UnitOfWork!
>
> Going down the road suggested above, the obvious way would be to
> implement services such as
> EntityBuilderFactory/QueryBuilderFactory/EntityFinder - and then more
> services to support new persistence/composites.
> The UOW can NOT depend directly on those, since it cannot know which
> sort of composites are persisted. So the services will have to manage
> that themselves, and need to implement a common interface for receiving
> notifications before/during/after completion of a UOW. The services
> could have a concern ensuring the existence of an active UOW.
>
> I believe it CAN be done - but would require a lot of collaboration
> between the UOW and those services - and a lot of the services would
> have to hold maps Map<UnitOfWork,"Stuff"> for stores and pending changes.
>
> Can it be improved?
>
> Currently all services are singletons (at least within a module) - that
> is, they have module scope. The services may implement interfaces
> allowing them to get notified when the module is activated/passivated.
>
> What if we had "UnitOfWork-scoped" services?
>
> A service with UOW scope is having a single instance per UOW (if used) -
> which is ensured by the runtime itself. No need for having a concern
> ensuring an active UOW - if there is no UOW there is simply no instance!
> References to a UOW-scoped service is simply a proxy delegating to the
> correct implementation at runtime - (the same pattern has been used in
> other frameworks, normally to delegate to a ThreadLocal instance). The
> service may implement interfaces allowing to get notified when the UOW
> is activated/passivated - maybe the usual Activation/Activator
> interfaces can be used?
>
> The point is that these services can be stateful - and could be used to
> hold eg. pending changes (entities/messages/events/aggregates) inside a
> UOW. No Maps necessary. No need for the UOW to do that book keeping. So
> EntityBuilderFactory/EntityFinder/QueryBuilderFactory could be
> UOW-scoped services. The services would not need to register themselves
> with the UOW (that is done automatically). And the UOW would not need to
> tell a lot of details on completion, since those details are remembered
> by the service itself.
>
> This can be used for persistance stuff but other ideas come to mind
>  - a service could implement a logging API, but only flush all debug
> messages regarding a UOW to disk in case of that UOW failing.
> - another service could track usage patterns of the application sorted
> by use cases.
>
> We could have a SpatialQueryBuilderFactory as a UOW-scoped service,
> using an underlying indexer
> We could have a SOLRQueryBuilderFactory as a UOW-scoped service using an
> underlying indexer.
> We could have a MessageDispatcher as a UOW-scoped service using a JMS
> implementation below
> .......
>
> Maybe we could even have services with other kind of scopes - the
> obvious case is HttpSession - but i guess we are now far beyond 3.0 scope=
.
>
> Am I making sense? Would it be worthwhile to attempt a prototype based
> on some of these ideas?
>
> /Kent
>
>


--=20
Niclas Hedhman, Software Developer
http://zest.apache.org - New Energy for Java

--089e0149422030eae5051e22c007--