polygene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jiri Jetmar <juergen.jet...@gmail.com>
Subject Re: GeoSpatial Query support
Date Thu, 11 Jun 2015 13:15:01 GMT
Evtl. here the usage of the spatial extension :

https://github.com/apache/zest-qi4j/blob/ramtej-fb/spatial.queries/core/testsupport/src/main/java/org/qi4j/test/indexing/AbstractSpatialRegressionTest.java

Cheers,
jj

2015-06-11 13:39 GMT+02:00 Niclas Hedhman <niclas@hedhman.org>:

> Ah, I see... This would allow the "Module" handling to be preserved if
> necessary, and makes sense to me.
>
> There is already a QueryBuilderSPI class, but not sure how it works (or is
> not in use).
>
>
> The Spatial Query stuff sits in a separate branch. See
>
>
> https://github.com/apache/zest-qi4j/tree/ramtej-fb/spatial.queries/core/api/src/main/java/org/qi4j/api/geometry
>
>
> https://github.com/apache/zest-qi4j/tree/ramtej-fb/spatial.queries/libraries/geometry
>
>
> https://github.com/apache/zest-qi4j/tree/ramtej-fb/spatial.queries/libraries/spatial
>
>
> https://github.com/apache/zest-qi4j/tree/ramtej-fb/spatial.queries/extensions/indexing-elasticsearch/src/main/java/org/qi4j/index/elasticsearch/extensions/spatial
>
> There might be something I forgot...
>
>
>
> On Thu, Jun 11, 2015 at 6:59 PM, Kent Sølvsten <kent.soelvsten@gmail.com>
> wrote:
>
> > Actually I was not thinking of QueryBuilderFactory as a decoupled
> > service, but rather as a SPI like EntityFinder.
> >
> > Something like
> >
> > /public interface SolrQueryService  extends EntityFinder,
> > StateChangeListener, QueryBuilderFactory, ServiceComposite//
> > //public interface SQLIndexingEngineService  extends
> > StateChangeListener, EntityFinder, ///QueryBuilderFactory,
> > /ServiceComposite//
> > /
> > If i get the current status correct, we have 4 indexing extensions:
> >
> > Solr: Supporting solr expressions
> > Sql: Supporting "specification" expressions
> > Rdf: Supporting "specification" expressions, sparql expressions and
> > serql expression
> > ES: Supporting "specification" expressions, not sure about "native
> queries"
> >
> > So I am thinking of each indexing extension having a QueryBuilderFactory
> > supporting 1-3 QueryBuilder APIs - or to turn it around:
> >
> > SolrQueryBuilder: Supported by Solr-indexer
> > SesameQueryBuilder (not sure if this should be split in 2): Supported by
> > Rdf-indexer
> > SpecificationQueryBuilder: Supported by multiple indexers.
> >
> > We could have a library for each QueryBuilder API - accessed by both
> > application code and relevant EntityStores.
> >
> > The implementation of SpecificationQueryBuilder is currently shared by
> > multiple indexers
> > - so we will either have to find a place to put the implementation code
> > (same library as the API or a separate library only used by indexers)
> > - or we might want to change the workflow a little bit (instead of
> > letting "SqlEntityFinder" convert a SpecificationQueryBuilderImpl to a
> > SQL expression we could consider using a "SqlSpecificationQueryBuilder"
> > instead).
> >
> > Finally there is the question of what a SpecificationQueryBuilder should
> > look like - can we have a single one to rule  them all" ?
> > - I have not seen the spatial stuff, but i guess it introduces both some
> > new types (points?, spheres?) and some new operators (near ? , within ?)
> > -  SpatialSpecificationQueryBuilder extends SpecificationQueryBuilder ?
> >
> > /Kent
> >
> >
> >
> > Den 11-06-2015 kl. 11:29 skrev Niclas Hedhman:
> > > Kent,
> > > Nice to hear your refreshing input. Perhaps you have a very strong
> point.
> > > Loooong ago, the Query and Storage was more intertwined (days before
> > > org.qi4j.api.property.Property and we tried to do pojo-style
> > "properties")
> > > than it is today, and perhaps we stopped that separation "too early"
> and
> > > even further modularization makes good sense. I am not sure...
> > >
> > > But let me guess; People tend to forget how complex the "Module" is.
> The
> > > "current module" is two parts, one that holds the module from where
> > > UnitOfWork is created, and one "follows" the execution stack, i.e. just
> > > before the invocation stack is invoked, a bunch of stuff is assigned to
> > it,
> > > including this handling of the "current module". Why? That is to make
> the
> > > whole Visibility thing to work as expected. In the early days, we had a
> > lot
> > > of "bugs" (unexpected behavior) due to this...
> > >
> > > I am guessing that a QueryBuilder @Service would have problem with the
> > same
> > > thing, i.e. it can handle the Entity type that is being requested, but
> > any
> > > dereferencing of associations might cause problems. As I said above, I
> am
> > > not sure, but willing to investigate it.
> > >
> > >
> > > Regarding your examples;
> > > SpecificationQueryBuilder<Person> sqb =
> > > this.module.newQueryBuilder( SpecificationQueryBuilder.class,
> > > Person.class );
> > >
> > > here you still have "this.module" but in the text I get the impression
> > that
> > > you even consider a totally stand-alone @Service QueryBuilderFactory
> > > (instead of the @Structure one), with regular injection.
> > >
> > >
> > > Cheers
> > >
> > > On Thu, Jun 11, 2015 at 3:20 PM, Kent Sølvsten <
> kent.soelvsten@gmail.com
> > >
> > > wrote:
> > >
> > >> While I agree that the goal of decoupling the query part from core is
> a
> > >> worthy one, I fail to see what UnitOfWorkFactory has to do with that.
> > >>
> > >> Decoupling the UnitOfWork may or may not make sence, but I think that
> is
> > >> another discussion - which could make sense eg. if you wish to
> integrate
> > >> a Zest application with an XA transaction manager.
> > >>
> > >> Since the EntityFinders are responsible for executing a Query it makes
> > >> since to me to let the EntityFinder be involved in creating the query.
> > >> So a possible design could be to simply break out the querybuilder
> API's
> > >> form core. No more, no less.
> > >>
> > >> Let us say that the core API only consists of
> > >>
> > >> public interface QueryBuilder<T> {
> > >>   Query<T> newQuery( UnitOfWork uoq );
> > >> }
> > >>
> > >> public interface QueryBuilderFactory {
> > >>     <QB,T> QB<T> newQueryBuilder( Class<QB extends QueryBuilder>
> > >> builderType, Class<T> resultType )
> > >>         throws MissingIndexingSystemException,
> > >> InvalidQueryLanguageException;
> > >> }
> > >>
> > >> Query interface still in core, looking as of today.
> > >>
> > >> QueryBuilderFactory should no longer be embedded inside the module,
> but
> > >> instead be a service (probably implemented by the extension that also
> > >> implements the EntityFinder).
> > >>
> > >> Concrete QueryBuilder interfaces/impls and Query implementations are
> > >> moved to libraries (one library implementing
> SpecificationQueryBuilder,
> > >> another implementing SparQLQueryBuilder ....)
> > >>
> > >> Application code could then be something like:
> > >>
> > >>         SpecificationQueryBuilder<Person> sqb =
> > >> this.module.newQueryBuilder( SpecificationQueryBuilder.class,
> > >> Person.class ); // h
> > >>         Property<String> nameProp = templateFor( Person.class
> ).name();
> > >>         sqb = sqb.where( eq( nameProp, "Kent") );
> > >>         Query<Person> query = unitOfWork.newQuery( qb );
> > >>
> > >> or
> > >>         SqlQueryBuilder<Person> sql = this.module.newQueryBuilder(
> > >> SqlQueryBuilder.class, Person.class );
> > >>         sql = sql.where( "name = 'Kent'");
> > >>         Query<Person> query = sql.newQuery( unitOfWork );  // this
> > >> QueryBuilder might have a special query implementation
> > >> or
> > >>         SesameQueryBuilder<Person> sesame =
> this.module.newQueryBuilder(
> > >> SesameQueryBuilder.class, Person.class );
> > >>         sesame = sesame.bySparql( queryString);  // this queryBuilder
> is
> > >> now probably immutable
> > >>         Query<Person> query = sesame.newQuery( unitOfWork );  //
this
> > >> QueryBuilder might have a special query implementation
> > >>
> > >>
> > >> Some of the QueryBuilder libraries might only consist of an API
> > >> (implementations supplied by the extension also supplying the
> > >> EntityFinder) - other libraries might have a default implementation to
> > >> facilitate several entitystores supporting that Query 'language'.
> > >> Note that some EntityFinders  might support several Query 'languages',
> > >> both a specification based and a native one.
> > >>
> > >> We might even some day be able to support JPA or Hibernate, allowing
> the
> > >> user to create a Zest app combining existing data in a relational
> > >> database with geospatial data? (that would probably require some
> > >> re-thinking of the UnitOfWork concept).
> > >>
> > >> Am I making sense?
> > >>
> > >> /Kent
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> Den 06-06-2015 kl. 05:41 skrev Niclas Hedhman:
> > >>> Thanks Martin, that gives a nice overview of the situation (you
> should
> > >>> plaster that on frontpage)...
> > >>>
> > >>> Zest gang,
> > >>> So, I think our challenge is to be able to introduce Geospatial
> > >>> indexing/querying, without introducing a dependency.
> > >>>
> > >>> What do I mean by this?
> > >>>
> > >>> Well, we already have another "Custom Query" type, the Lucene
> free-text
> > >>> search, which couldn't be fitted nicely into the very nature of the
> > >>> UnitOfWork/QueryExpressions/QueryBuilder system.
> > >>>
> > >>> So let's hypothesize about;
> > >>>   * Ability to introduce new indexable data types,
> > >>>   * Ability to introduce extensions to the Query DSL
> > >>>   * Ability to extend the Indexing/Query extension itself.
> > >>>
> > >>>
> > >>> Where is the root of this system? Well, it all begins in the
> > UnitOfWork.
> > >>> So, one possibility would be to disconnect UnitOfWorkFactory from the
> > >>> Module itself, and have a ServiceComposite for the UnitOfWorkFactory.
> > >> Once
> > >>> the UnifOfWorkFactory implementation is outside the Core Runtime, so
> is
> > >>> UnitOfWork and everything that derives from it. It SHOULD mean that
> all
> > >>> parts could be Composites, in which case we can add arbitrary methods
> > to
> > >>> them, the static methods of QueryExpressions could go away and
> probably
> > >>> other, yet to be discovered, benefits.
> > >>>
> > >>> Granted, it won't be common for users to create their own, but having
> > >> this
> > >>> possibility, without becoming incompatible and without introducing
> > >>> dependencies in the Core, seems to me to be worthwhile going down
> this
> > >>> route. Having such a major part of the Core to be on the
> > "Composite-side
> > >> of
> > >>> things", might open up more cool ideas...
> > >>> It also feels "right" along our habit of breaking things out of Core
> > and
> > >>> moving towards more modularity.
> > >>>
> > >>> On paper, it sounds reasonably easy to do this, but I bet the devil
> is
> > in
> > >>> the details. Maybe an impossible circular dependency will arise, or
> > >>> something to that extent.
> > >>>
> > >>>
> > >>> Any thought on this?
> > >>>
> > >>>
> > >>> Cheers
> > >>>
> > >>> On Fri, Jun 5, 2015 at 4:57 PM, Martin Desruisseaux <
> > >>> martin.desruisseaux@geomatys.com> wrote:
> > >>>
> > >>>> Thanks Chris for getting Zest and SIS in touch. I just finished
> > reading
> > >>>> the thread. There is some tips for information purpose:
> > >>>>
> > >>>> Niclas Hedhman wrote:
> > >>>>
> > >>>>> So, IF SIS are primarily based around interfaces, then that
would
> be
> > >>>>> great and we can possibly leverage quite a bit, especially
at what
> we
> > >>>>> call "Library" level, i.e. not part of the Core runtime itself,
> which
> > >>>>> we try to keep free of dependencies
> > >>>> The core part of SIS is defined by a set of interfaces provided
by a
> > >>>> separated project: http://www.geoapi.org/. GeoAPI consists of only
> > >>>> interfaces, except some classes for Exception, Enum and "CodeList"
> > >>>> (similar to Enum but extensible). GeoAPI is based on some
> > international
> > >>>> standards published jointly by the Open Geospatial Consortium (OGC)
> > and
> > >>>> International Organization for Standardization (ISO). It currently
> > >>>> covers only a small part of OGC standards, but this includes map
> > >>>> projections.
> > >>>>
> > >>>> Apache SIS is a GeoAPI 3.0 implementation. The GeoAPI project
> provides
> > >>>> also implementations as wrappers around Proj.4 (the C/C++ library
> used
> > >>>> by GDAL) and the UCAR NetCDF library. All those projects have
> > advantages
> > >>>> and inconvenient (e.g. Proj.4 supports a wider range of map
> > projections
> > >>>> than SIS, but SIS is more compliant with OGC/ISO standards). But
if
> > Zest
> > >>>> depends directly on only GeoAPI interfaces you would have the
> freedom
> > to
> > >>>> change implementation. However I do not know if Spatial4J would
be
> > >>>> interested to implement GeoAPI interfaces.
> > >>>>
> > >>>> On geometry and indexing, there is no satisfying solution on GeoAPI
> > side
> > >>>> yet. In particular, geometries are defined by the ISO 19107
> > >>>> international standard, which is currently under revision. This
will
> > >>>> have a deep impact on GeoAPI once the ISO revision will be
> completed.
> > >>>>
> > >>>> Please let us know if you would like to explore further (e.g. how
to
> > >>>> apply a map projection using the API defined by GeoAPI).
> > >>>>
> > >>>>     Martin
> > >>>>
> > >>>>
> > >>
> > >
> >
> >
>
>
> --
> Niclas Hedhman, Software Developer
> http://zest.apache.org - New Energy for Java
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message