polygene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kent Sølvsten <kent.soelvs...@gmail.com>
Subject Re: GeoSpatial Query support
Date Thu, 11 Jun 2015 10:59:28 GMT
Actually I was not thinking of QueryBuilderFactory as a decoupled
service, but rather as a SPI like EntityFinder.

Something like

/public interface SolrQueryService  extends EntityFinder,
StateChangeListener, QueryBuilderFactory, ServiceComposite//
//public interface SQLIndexingEngineService  extends
StateChangeListener, EntityFinder, ///QueryBuilderFactory,
/ServiceComposite//
/
If i get the current status correct, we have 4 indexing extensions:

Solr: Supporting solr expressions
Sql: Supporting "specification" expressions
Rdf: Supporting "specification" expressions, sparql expressions and
serql expression
ES: Supporting "specification" expressions, not sure about "native queries"

So I am thinking of each indexing extension having a QueryBuilderFactory
supporting 1-3 QueryBuilder APIs - or to turn it around:

SolrQueryBuilder: Supported by Solr-indexer
SesameQueryBuilder (not sure if this should be split in 2): Supported by
Rdf-indexer
SpecificationQueryBuilder: Supported by multiple indexers.

We could have a library for each QueryBuilder API - accessed by both
application code and relevant EntityStores.

The implementation of SpecificationQueryBuilder is currently shared by
multiple indexers
- so we will either have to find a place to put the implementation code
(same library as the API or a separate library only used by indexers)
- or we might want to change the workflow a little bit (instead of
letting "SqlEntityFinder" convert a SpecificationQueryBuilderImpl to a
SQL expression we could consider using a "SqlSpecificationQueryBuilder"
instead).

Finally there is the question of what a SpecificationQueryBuilder should
look like - can we have a single one to rule  them all" ?
- I have not seen the spatial stuff, but i guess it introduces both some
new types (points?, spheres?) and some new operators (near ? , within ?)
-  SpatialSpecificationQueryBuilder extends SpecificationQueryBuilder ?

/Kent



Den 11-06-2015 kl. 11:29 skrev Niclas Hedhman:
> Kent,
> Nice to hear your refreshing input. Perhaps you have a very strong point.
> Loooong ago, the Query and Storage was more intertwined (days before
> org.qi4j.api.property.Property and we tried to do pojo-style "properties")
> than it is today, and perhaps we stopped that separation "too early" and
> even further modularization makes good sense. I am not sure...
>
> But let me guess; People tend to forget how complex the "Module" is. The
> "current module" is two parts, one that holds the module from where
> UnitOfWork is created, and one "follows" the execution stack, i.e. just
> before the invocation stack is invoked, a bunch of stuff is assigned to it,
> including this handling of the "current module". Why? That is to make the
> whole Visibility thing to work as expected. In the early days, we had a lot
> of "bugs" (unexpected behavior) due to this...
>
> I am guessing that a QueryBuilder @Service would have problem with the same
> thing, i.e. it can handle the Entity type that is being requested, but any
> dereferencing of associations might cause problems. As I said above, I am
> not sure, but willing to investigate it.
>
>
> Regarding your examples;
> SpecificationQueryBuilder<Person> sqb =
> this.module.newQueryBuilder( SpecificationQueryBuilder.class,
> Person.class );
>
> here you still have "this.module" but in the text I get the impression that
> you even consider a totally stand-alone @Service QueryBuilderFactory
> (instead of the @Structure one), with regular injection.
>
>
> Cheers
>
> On Thu, Jun 11, 2015 at 3:20 PM, Kent Sølvsten <kent.soelvsten@gmail.com>
> wrote:
>
>> While I agree that the goal of decoupling the query part from core is a
>> worthy one, I fail to see what UnitOfWorkFactory has to do with that.
>>
>> Decoupling the UnitOfWork may or may not make sence, but I think that is
>> another discussion - which could make sense eg. if you wish to integrate
>> a Zest application with an XA transaction manager.
>>
>> Since the EntityFinders are responsible for executing a Query it makes
>> since to me to let the EntityFinder be involved in creating the query.
>> So a possible design could be to simply break out the querybuilder API's
>> form core. No more, no less.
>>
>> Let us say that the core API only consists of
>>
>> public interface QueryBuilder<T> {
>>   Query<T> newQuery( UnitOfWork uoq );
>> }
>>
>> public interface QueryBuilderFactory {
>>     <QB,T> QB<T> newQueryBuilder( Class<QB extends QueryBuilder>
>> builderType, Class<T> resultType )
>>         throws MissingIndexingSystemException,
>> InvalidQueryLanguageException;
>> }
>>
>> Query interface still in core, looking as of today.
>>
>> QueryBuilderFactory should no longer be embedded inside the module, but
>> instead be a service (probably implemented by the extension that also
>> implements the EntityFinder).
>>
>> Concrete QueryBuilder interfaces/impls and Query implementations are
>> moved to libraries (one library implementing SpecificationQueryBuilder,
>> another implementing SparQLQueryBuilder ....)
>>
>> Application code could then be something like:
>>
>>         SpecificationQueryBuilder<Person> sqb =
>> this.module.newQueryBuilder( SpecificationQueryBuilder.class,
>> Person.class ); // h
>>         Property<String> nameProp = templateFor( Person.class ).name();
>>         sqb = sqb.where( eq( nameProp, "Kent") );
>>         Query<Person> query = unitOfWork.newQuery( qb );
>>
>> or
>>         SqlQueryBuilder<Person> sql = this.module.newQueryBuilder(
>> SqlQueryBuilder.class, Person.class );
>>         sql = sql.where( "name = 'Kent'");
>>         Query<Person> query = sql.newQuery( unitOfWork );  // this
>> QueryBuilder might have a special query implementation
>> or
>>         SesameQueryBuilder<Person> sesame = this.module.newQueryBuilder(
>> SesameQueryBuilder.class, Person.class );
>>         sesame = sesame.bySparql( queryString);  // this queryBuilder is
>> now probably immutable
>>         Query<Person> query = sesame.newQuery( unitOfWork );  // this
>> QueryBuilder might have a special query implementation
>>
>>
>> Some of the QueryBuilder libraries might only consist of an API
>> (implementations supplied by the extension also supplying the
>> EntityFinder) - other libraries might have a default implementation to
>> facilitate several entitystores supporting that Query 'language'.
>> Note that some EntityFinders  might support several Query 'languages',
>> both a specification based and a native one.
>>
>> We might even some day be able to support JPA or Hibernate, allowing the
>> user to create a Zest app combining existing data in a relational
>> database with geospatial data? (that would probably require some
>> re-thinking of the UnitOfWork concept).
>>
>> Am I making sense?
>>
>> /Kent
>>
>>
>>
>>
>>
>>
>>
>>
>> Den 06-06-2015 kl. 05:41 skrev Niclas Hedhman:
>>> Thanks Martin, that gives a nice overview of the situation (you should
>>> plaster that on frontpage)...
>>>
>>> Zest gang,
>>> So, I think our challenge is to be able to introduce Geospatial
>>> indexing/querying, without introducing a dependency.
>>>
>>> What do I mean by this?
>>>
>>> Well, we already have another "Custom Query" type, the Lucene free-text
>>> search, which couldn't be fitted nicely into the very nature of the
>>> UnitOfWork/QueryExpressions/QueryBuilder system.
>>>
>>> So let's hypothesize about;
>>>   * Ability to introduce new indexable data types,
>>>   * Ability to introduce extensions to the Query DSL
>>>   * Ability to extend the Indexing/Query extension itself.
>>>
>>>
>>> Where is the root of this system? Well, it all begins in the UnitOfWork.
>>> So, one possibility would be to disconnect UnitOfWorkFactory from the
>>> Module itself, and have a ServiceComposite for the UnitOfWorkFactory.
>> Once
>>> the UnifOfWorkFactory implementation is outside the Core Runtime, so is
>>> UnitOfWork and everything that derives from it. It SHOULD mean that all
>>> parts could be Composites, in which case we can add arbitrary methods to
>>> them, the static methods of QueryExpressions could go away and probably
>>> other, yet to be discovered, benefits.
>>>
>>> Granted, it won't be common for users to create their own, but having
>> this
>>> possibility, without becoming incompatible and without introducing
>>> dependencies in the Core, seems to me to be worthwhile going down this
>>> route. Having such a major part of the Core to be on the "Composite-side
>> of
>>> things", might open up more cool ideas...
>>> It also feels "right" along our habit of breaking things out of Core and
>>> moving towards more modularity.
>>>
>>> On paper, it sounds reasonably easy to do this, but I bet the devil is in
>>> the details. Maybe an impossible circular dependency will arise, or
>>> something to that extent.
>>>
>>>
>>> Any thought on this?
>>>
>>>
>>> Cheers
>>>
>>> On Fri, Jun 5, 2015 at 4:57 PM, Martin Desruisseaux <
>>> martin.desruisseaux@geomatys.com> wrote:
>>>
>>>> Thanks Chris for getting Zest and SIS in touch. I just finished reading
>>>> the thread. There is some tips for information purpose:
>>>>
>>>> Niclas Hedhman wrote:
>>>>
>>>>> So, IF SIS are primarily based around interfaces, then that would be
>>>>> great and we can possibly leverage quite a bit, especially at what we
>>>>> call "Library" level, i.e. not part of the Core runtime itself, which
>>>>> we try to keep free of dependencies
>>>> The core part of SIS is defined by a set of interfaces provided by a
>>>> separated project: http://www.geoapi.org/. GeoAPI consists of only
>>>> interfaces, except some classes for Exception, Enum and "CodeList"
>>>> (similar to Enum but extensible). GeoAPI is based on some international
>>>> standards published jointly by the Open Geospatial Consortium (OGC) and
>>>> International Organization for Standardization (ISO). It currently
>>>> covers only a small part of OGC standards, but this includes map
>>>> projections.
>>>>
>>>> Apache SIS is a GeoAPI 3.0 implementation. The GeoAPI project provides
>>>> also implementations as wrappers around Proj.4 (the C/C++ library used
>>>> by GDAL) and the UCAR NetCDF library. All those projects have advantages
>>>> and inconvenient (e.g. Proj.4 supports a wider range of map projections
>>>> than SIS, but SIS is more compliant with OGC/ISO standards). But if Zest
>>>> depends directly on only GeoAPI interfaces you would have the freedom to
>>>> change implementation. However I do not know if Spatial4J would be
>>>> interested to implement GeoAPI interfaces.
>>>>
>>>> On geometry and indexing, there is no satisfying solution on GeoAPI side
>>>> yet. In particular, geometries are defined by the ISO 19107
>>>> international standard, which is currently under revision. This will
>>>> have a deep impact on GeoAPI once the ISO revision will be completed.
>>>>
>>>> Please let us know if you would like to explore further (e.g. how to
>>>> apply a map projection using the API defined by GeoAPI).
>>>>
>>>>     Martin
>>>>
>>>>
>>
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message