polygene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niclas Hedhman <nic...@hedhman.org>
Subject Re: GeoSpatial Query support
Date Thu, 11 Jun 2015 11:39:18 GMT
Ah, I see... This would allow the "Module" handling to be preserved if
necessary, and makes sense to me.

There is already a QueryBuilderSPI class, but not sure how it works (or is
not in use).


The Spatial Query stuff sits in a separate branch. See

https://github.com/apache/zest-qi4j/tree/ramtej-fb/spatial.queries/core/api/src/main/java/org/qi4j/api/geometry

https://github.com/apache/zest-qi4j/tree/ramtej-fb/spatial.queries/libraries/geometry

https://github.com/apache/zest-qi4j/tree/ramtej-fb/spatial.queries/libraries/spatial

https://github.com/apache/zest-qi4j/tree/ramtej-fb/spatial.queries/extensions/indexing-elasticsearch/src/main/java/org/qi4j/index/elasticsearch/extensions/spatial

There might be something I forgot...



On Thu, Jun 11, 2015 at 6:59 PM, Kent Sølvsten <kent.soelvsten@gmail.com>
wrote:

> Actually I was not thinking of QueryBuilderFactory as a decoupled
> service, but rather as a SPI like EntityFinder.
>
> Something like
>
> /public interface SolrQueryService  extends EntityFinder,
> StateChangeListener, QueryBuilderFactory, ServiceComposite//
> //public interface SQLIndexingEngineService  extends
> StateChangeListener, EntityFinder, ///QueryBuilderFactory,
> /ServiceComposite//
> /
> If i get the current status correct, we have 4 indexing extensions:
>
> Solr: Supporting solr expressions
> Sql: Supporting "specification" expressions
> Rdf: Supporting "specification" expressions, sparql expressions and
> serql expression
> ES: Supporting "specification" expressions, not sure about "native queries"
>
> So I am thinking of each indexing extension having a QueryBuilderFactory
> supporting 1-3 QueryBuilder APIs - or to turn it around:
>
> SolrQueryBuilder: Supported by Solr-indexer
> SesameQueryBuilder (not sure if this should be split in 2): Supported by
> Rdf-indexer
> SpecificationQueryBuilder: Supported by multiple indexers.
>
> We could have a library for each QueryBuilder API - accessed by both
> application code and relevant EntityStores.
>
> The implementation of SpecificationQueryBuilder is currently shared by
> multiple indexers
> - so we will either have to find a place to put the implementation code
> (same library as the API or a separate library only used by indexers)
> - or we might want to change the workflow a little bit (instead of
> letting "SqlEntityFinder" convert a SpecificationQueryBuilderImpl to a
> SQL expression we could consider using a "SqlSpecificationQueryBuilder"
> instead).
>
> Finally there is the question of what a SpecificationQueryBuilder should
> look like - can we have a single one to rule  them all" ?
> - I have not seen the spatial stuff, but i guess it introduces both some
> new types (points?, spheres?) and some new operators (near ? , within ?)
> -  SpatialSpecificationQueryBuilder extends SpecificationQueryBuilder ?
>
> /Kent
>
>
>
> Den 11-06-2015 kl. 11:29 skrev Niclas Hedhman:
> > Kent,
> > Nice to hear your refreshing input. Perhaps you have a very strong point.
> > Loooong ago, the Query and Storage was more intertwined (days before
> > org.qi4j.api.property.Property and we tried to do pojo-style
> "properties")
> > than it is today, and perhaps we stopped that separation "too early" and
> > even further modularization makes good sense. I am not sure...
> >
> > But let me guess; People tend to forget how complex the "Module" is. The
> > "current module" is two parts, one that holds the module from where
> > UnitOfWork is created, and one "follows" the execution stack, i.e. just
> > before the invocation stack is invoked, a bunch of stuff is assigned to
> it,
> > including this handling of the "current module". Why? That is to make the
> > whole Visibility thing to work as expected. In the early days, we had a
> lot
> > of "bugs" (unexpected behavior) due to this...
> >
> > I am guessing that a QueryBuilder @Service would have problem with the
> same
> > thing, i.e. it can handle the Entity type that is being requested, but
> any
> > dereferencing of associations might cause problems. As I said above, I am
> > not sure, but willing to investigate it.
> >
> >
> > Regarding your examples;
> > SpecificationQueryBuilder<Person> sqb =
> > this.module.newQueryBuilder( SpecificationQueryBuilder.class,
> > Person.class );
> >
> > here you still have "this.module" but in the text I get the impression
> that
> > you even consider a totally stand-alone @Service QueryBuilderFactory
> > (instead of the @Structure one), with regular injection.
> >
> >
> > Cheers
> >
> > On Thu, Jun 11, 2015 at 3:20 PM, Kent Sølvsten <kent.soelvsten@gmail.com
> >
> > wrote:
> >
> >> While I agree that the goal of decoupling the query part from core is a
> >> worthy one, I fail to see what UnitOfWorkFactory has to do with that.
> >>
> >> Decoupling the UnitOfWork may or may not make sence, but I think that is
> >> another discussion - which could make sense eg. if you wish to integrate
> >> a Zest application with an XA transaction manager.
> >>
> >> Since the EntityFinders are responsible for executing a Query it makes
> >> since to me to let the EntityFinder be involved in creating the query.
> >> So a possible design could be to simply break out the querybuilder API's
> >> form core. No more, no less.
> >>
> >> Let us say that the core API only consists of
> >>
> >> public interface QueryBuilder<T> {
> >>   Query<T> newQuery( UnitOfWork uoq );
> >> }
> >>
> >> public interface QueryBuilderFactory {
> >>     <QB,T> QB<T> newQueryBuilder( Class<QB extends QueryBuilder>
> >> builderType, Class<T> resultType )
> >>         throws MissingIndexingSystemException,
> >> InvalidQueryLanguageException;
> >> }
> >>
> >> Query interface still in core, looking as of today.
> >>
> >> QueryBuilderFactory should no longer be embedded inside the module, but
> >> instead be a service (probably implemented by the extension that also
> >> implements the EntityFinder).
> >>
> >> Concrete QueryBuilder interfaces/impls and Query implementations are
> >> moved to libraries (one library implementing SpecificationQueryBuilder,
> >> another implementing SparQLQueryBuilder ....)
> >>
> >> Application code could then be something like:
> >>
> >>         SpecificationQueryBuilder<Person> sqb =
> >> this.module.newQueryBuilder( SpecificationQueryBuilder.class,
> >> Person.class ); // h
> >>         Property<String> nameProp = templateFor( Person.class ).name();
> >>         sqb = sqb.where( eq( nameProp, "Kent") );
> >>         Query<Person> query = unitOfWork.newQuery( qb );
> >>
> >> or
> >>         SqlQueryBuilder<Person> sql = this.module.newQueryBuilder(
> >> SqlQueryBuilder.class, Person.class );
> >>         sql = sql.where( "name = 'Kent'");
> >>         Query<Person> query = sql.newQuery( unitOfWork );  // this
> >> QueryBuilder might have a special query implementation
> >> or
> >>         SesameQueryBuilder<Person> sesame = this.module.newQueryBuilder(
> >> SesameQueryBuilder.class, Person.class );
> >>         sesame = sesame.bySparql( queryString);  // this queryBuilder is
> >> now probably immutable
> >>         Query<Person> query = sesame.newQuery( unitOfWork );  // this
> >> QueryBuilder might have a special query implementation
> >>
> >>
> >> Some of the QueryBuilder libraries might only consist of an API
> >> (implementations supplied by the extension also supplying the
> >> EntityFinder) - other libraries might have a default implementation to
> >> facilitate several entitystores supporting that Query 'language'.
> >> Note that some EntityFinders  might support several Query 'languages',
> >> both a specification based and a native one.
> >>
> >> We might even some day be able to support JPA or Hibernate, allowing the
> >> user to create a Zest app combining existing data in a relational
> >> database with geospatial data? (that would probably require some
> >> re-thinking of the UnitOfWork concept).
> >>
> >> Am I making sense?
> >>
> >> /Kent
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >> Den 06-06-2015 kl. 05:41 skrev Niclas Hedhman:
> >>> Thanks Martin, that gives a nice overview of the situation (you should
> >>> plaster that on frontpage)...
> >>>
> >>> Zest gang,
> >>> So, I think our challenge is to be able to introduce Geospatial
> >>> indexing/querying, without introducing a dependency.
> >>>
> >>> What do I mean by this?
> >>>
> >>> Well, we already have another "Custom Query" type, the Lucene free-text
> >>> search, which couldn't be fitted nicely into the very nature of the
> >>> UnitOfWork/QueryExpressions/QueryBuilder system.
> >>>
> >>> So let's hypothesize about;
> >>>   * Ability to introduce new indexable data types,
> >>>   * Ability to introduce extensions to the Query DSL
> >>>   * Ability to extend the Indexing/Query extension itself.
> >>>
> >>>
> >>> Where is the root of this system? Well, it all begins in the
> UnitOfWork.
> >>> So, one possibility would be to disconnect UnitOfWorkFactory from the
> >>> Module itself, and have a ServiceComposite for the UnitOfWorkFactory.
> >> Once
> >>> the UnifOfWorkFactory implementation is outside the Core Runtime, so is
> >>> UnitOfWork and everything that derives from it. It SHOULD mean that all
> >>> parts could be Composites, in which case we can add arbitrary methods
> to
> >>> them, the static methods of QueryExpressions could go away and probably
> >>> other, yet to be discovered, benefits.
> >>>
> >>> Granted, it won't be common for users to create their own, but having
> >> this
> >>> possibility, without becoming incompatible and without introducing
> >>> dependencies in the Core, seems to me to be worthwhile going down this
> >>> route. Having such a major part of the Core to be on the
> "Composite-side
> >> of
> >>> things", might open up more cool ideas...
> >>> It also feels "right" along our habit of breaking things out of Core
> and
> >>> moving towards more modularity.
> >>>
> >>> On paper, it sounds reasonably easy to do this, but I bet the devil is
> in
> >>> the details. Maybe an impossible circular dependency will arise, or
> >>> something to that extent.
> >>>
> >>>
> >>> Any thought on this?
> >>>
> >>>
> >>> Cheers
> >>>
> >>> On Fri, Jun 5, 2015 at 4:57 PM, Martin Desruisseaux <
> >>> martin.desruisseaux@geomatys.com> wrote:
> >>>
> >>>> Thanks Chris for getting Zest and SIS in touch. I just finished
> reading
> >>>> the thread. There is some tips for information purpose:
> >>>>
> >>>> Niclas Hedhman wrote:
> >>>>
> >>>>> So, IF SIS are primarily based around interfaces, then that would
be
> >>>>> great and we can possibly leverage quite a bit, especially at what
we
> >>>>> call "Library" level, i.e. not part of the Core runtime itself,
which
> >>>>> we try to keep free of dependencies
> >>>> The core part of SIS is defined by a set of interfaces provided by a
> >>>> separated project: http://www.geoapi.org/. GeoAPI consists of only
> >>>> interfaces, except some classes for Exception, Enum and "CodeList"
> >>>> (similar to Enum but extensible). GeoAPI is based on some
> international
> >>>> standards published jointly by the Open Geospatial Consortium (OGC)
> and
> >>>> International Organization for Standardization (ISO). It currently
> >>>> covers only a small part of OGC standards, but this includes map
> >>>> projections.
> >>>>
> >>>> Apache SIS is a GeoAPI 3.0 implementation. The GeoAPI project provides
> >>>> also implementations as wrappers around Proj.4 (the C/C++ library used
> >>>> by GDAL) and the UCAR NetCDF library. All those projects have
> advantages
> >>>> and inconvenient (e.g. Proj.4 supports a wider range of map
> projections
> >>>> than SIS, but SIS is more compliant with OGC/ISO standards). But if
> Zest
> >>>> depends directly on only GeoAPI interfaces you would have the freedom
> to
> >>>> change implementation. However I do not know if Spatial4J would be
> >>>> interested to implement GeoAPI interfaces.
> >>>>
> >>>> On geometry and indexing, there is no satisfying solution on GeoAPI
> side
> >>>> yet. In particular, geometries are defined by the ISO 19107
> >>>> international standard, which is currently under revision. This will
> >>>> have a deep impact on GeoAPI once the ISO revision will be completed.
> >>>>
> >>>> Please let us know if you would like to explore further (e.g. how to
> >>>> apply a map projection using the API defined by GeoAPI).
> >>>>
> >>>>     Martin
> >>>>
> >>>>
> >>
> >
>
>


-- 
Niclas Hedhman, Software Developer
http://zest.apache.org - New Energy for Java

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message