polygene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jiri Jetmar <juergen.jet...@gmail.com>
Subject Re: GeoSpatial Query support
Date Wed, 03 Jun 2015 20:35:08 GMT
Hi Adam & Chris,

some notes on the Zest spatial implementations. It is in general structured
in three main components :

1.) Zest core that defines spatial values, like TPoint, TPolygon, ... as
well as the query DSL that allows to define spatial queries for the
underlying indexing system. The values are derived from the
http://geojson.org/ spec. The Zest spatial core is dependency free, e.g. no
3th party libraries are used.

2.) The indexing system that is capable to i.) index spatial values and
ii.)  "executes" the spatial queries. For now we are supporting
ElasticSearch for spatial queries. Means we are delegating the query to ES
while "traversing" the ES native query with some required metadata. There
are a number of "nasty" things while using ElasticSearch for spatial
queries like the "mappings", where one has to declare spatial types during
indexing, define how to index values (as Points or as Shapes) but this
is encapsulated by the Zest ES spatial extension. Also one has to deal with
projections, as ES just supports WGS84. Means when your query is using a
different projection, then we have to convert it to WGS84 while consider
the conversion accuracy..

3.) Supporting library that contains the conversion and projection code -
we are using here the spatial4j library.

Sample to create a spatial value :

       TPolygon polygon = TPolygon(module)
                .shell
                        (
                                new double[][]
                                        {
                                                {0, 0},
                                                {3, 0},
                                                {1, 3},
                                                {0, 3},
                                                {0, 0}
                                        }
                        )
                .withHoles(
                        TLinearRing(module).ring(new double[][]
                                {
                                        {0, 0},
                                        {1, 0},
                                        {1, 1},
                                        {0, 1},
                                        {0, 0}

                                }).geometry()
                        , TLinearRing(module).ring(new double[][]
                                {
                                        {0, 0},
                                        {2, 0},
                                        {2, 2},
                                        {0, 2},
                                        {0, 0}
                                }).geometry()
                )
                .geometry();

Sample spatial query :

        Query<A> query = unitOfWork.newQuery(
                qb
                        .where(and(
                                        *ST_Disjoint*
                                                (

templateFor(A.class).point(),

TPoint(module).y(2389280.7514562616).x(1286436.5975464052).geometry("
*EPSG:27572*"),
                                                        10, TUnit.METER
                                                ),
                                        *ST_Within*
                                                (

templateFor(A.class).point(),

TPoint(module).y(48.13905780942574).x(11.57958984375).*geometry(*),
                                                        100,TUnit.KILOMETER
                                                )
                                )
                        ));
        query.find();

Here we are mixing two different projections in one query. Default
projection is defined to be "EPSG:4326".
ES is using geohashes or quadtree approach to index spatial values, for now
we are using the quadtree approach. The required precision level (accuracy)
can be defined
in a configuration file :

spatial = { \
  \
  Enabled : true, \
  \
  Indexer=\
   { \
    Method=\
      { \
        Type      = "GEO_POINT", \ <-- ES spatial indexing method
(GEO_POINT or GEO_SHAPE)
        Precision = 1m, \  * <-- Indexing accuracy aka quadtree levels*
      }, \
    Projection=\
      { \
*        ConversionEnabled  = true, \*
*        ConversionAccuracy = 2m, \ <-- projection conversion accuracy
while indexing*
      }, \
   }, \
  \
  \
  Finder=\
    { \
    Projection=\
      { \
        ConversionEnabled  = true, \
        ConversionAccuracy = 2m, \ *<-- projection conversion accuracy
during query execution*
      } \
    } \
  }

Overall status:

Ok, the Zest Spatial stuff is for now very simple. We support the following
query expressions :

- ST_Within
- ST_Disjoint
- ST_Intersects

Due to implementation difficulties on ES I;m using a kind of "supported"
matrix that declares what combinations are allowed :

          // ST_Within
        supports(enable(ST_WithinSpecification.class),
propertyOf(AnyGeometry), filterOf(TPoint.class, TPolygon.class),
enable(OrderBy), SpatialConfiguration.INDEXING_METHOD.GEO_POINT);
        supports(enable(ST_WithinSpecification.class),
propertyOf(AnyGeometry), filterOf(TPoint.class, TPolygon.class),
disable(OrderBy), SpatialConfiguration.INDEXING_METHOD.GEO_SHAPE);

        // ST_Disjoint
        supports(disable(ST_DisjointSpecification.class),
propertyOf(AnyGeometry), filterOf(TPoint.class, TPolygon.class),
enable(OrderBy), SpatialConfiguration.INDEXING_METHOD.GEO_POINT);
        supports(enable(ST_DisjointSpecification.class),
propertyOf(AnyGeometry), filterOf(TPoint.class, TPolygon.class),
disable(OrderBy), SpatialConfiguration.INDEXING_METHOD.GEO_SHAPE);


        // ST_Intersects
        supports(disable(ST_IntersectsSpecification.class),
propertyOf(AnyGeometry), filterOf(TPoint.class, TPolygon.class),
enable(OrderBy), SpatialConfiguration.INDEXING_METHOD.GEO_POINT);
        supports(enable(ST_IntersectsSpecification.class),
propertyOf(AnyGeometry), filterOf(TPoint.class, TPolygon.class),
disable(OrderBy), SpatialConfiguration.INDEXING_METHOD.GEO_SHAPE);

Sample :

        supports(enable(ST_WithinSpecification.class),
propertyOf(AnyGeometry), filterOf(TPoint.class, TPolygon.class),
disable(OrderBy), SpatialConfiguration.INDEXING_METHOD.GEO_SHAPE);

== Enable support for *ST_With *on *AnyGeometry* and allow *TPoint*,
*TPolygon* as filter and *disable ordering* when *GEO_SHAPING* indexing
method is used.

This is indeed not a perfect solution, but using that approach we are able
to express the supported constellations and are able to generate a
"speaking" exception when
a unsupported query is defined.

Stability :

I can not say a lot on long-term stability yet. But I take the
OpenStreetMap Planet.osm and "pump" it to Apache ZEST and did some
concurrent queries on it
without any serious issues. In general ES performs really great on spatial
data !

Cheers,
Jiri








2015-06-03 17:12 GMT+02:00 <estrada.adam@gmail.com>:

> Most excellent, Dr. Mattmann! @sis, we should definitely collaborate with
> @zest on this one. If they are implementing simple query functionality,
> maybe they can take advantage of the work Marc has already done and also
> use the projection support Martin has been committing.
>
> Thoughts?
> Adam
>
> Sent from my iPhone
>
> > On Jun 3, 2015, at 11:00 AM, Mattmann, Chris A (3980) <
> chris.a.mattmann@jpl.nasa.gov> wrote:
> >
> > Maybe this is a good integration point for sis.apache.org?
> >
> > CC’ing the Apache SIS list for visibility
> >
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > Chris Mattmann, Ph.D.
> > Chief Architect
> > Instrument Software and Science Data Systems Section (398)
> > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> > Office: 168-519, Mailstop: 168-527
> > Email: chris.a.mattmann@nasa.gov
> > WWW:  http://sunset.usc.edu/~mattmann/
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > Adjunct Associate Professor, Computer Science Department
> > University of Southern California, Los Angeles, CA 90089 USA
> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> >
> >
> >
> >
> > -----Original Message-----
> > From: Jiri Jetmar <juergen.jetmar@gmail.com>
> > Reply-To: "dev@zest.apache.org" <dev@zest.apache.org>
> > Date: Wednesday, June 3, 2015 at 7:45 AM
> > To: "dev@zest.apache.org" <dev@zest.apache.org>
> > Subject: Re: GeoSpatial Query support
> >
> >> Hi Niclas,
> >>
> >> this ST_XX follows the http://en.wikipedia.org/wiki/DE-9IM and
> >> http://en.wikipedia.org/wiki/Simple_Features standards that are indeed
> >> "SQL" related. I used this expression to be similar with e.g. PostGIS,
> >> means when someone knows how to use spatial expressions on PostGIS,
> >> he should be able to understand the Apache Zest approach.
> >>
> >> But clearly I can "lowercase" the ST_Xx expressions in the code.
> >>
> >> Cheers,
> >> jj
> >>
> >> 2015-06-03 16:22 GMT+02:00 Niclas Hedhman <niclas@hedhman.org>:
> >>
> >>> Jiri,
> >>> There is one thing that bothers me; ST_Within and similar doesn't
> follow
> >>> the Java convention at all. I have researched and seen that many DBs
> >>> uses
> >>> this, but don't you think we should lower case st_ instead for methods?
> >>>
> >>>
> >>> Cheers
> >>> --
> >>> Niclas Hedhman, Software Developer
> >>> http://zest.apache.org - New Energy for Java
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message