polygene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (3980)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: GeoSpatial Query support
Date Thu, 04 Jun 2015 05:39:48 GMT
Thanks Jiri. Since Apache SIS handles a lot of the below,
potentially maybe there is room to start to integrate SIS into
Zest?

MartinD from SIS I think will speak up and help explain :)

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++




-----Original Message-----
From: Jiri Jetmar <juergen.jetmar@gmail.com>
Reply-To: "dev@zest.apache.org" <dev@zest.apache.org>
Date: Wednesday, June 3, 2015 at 1:35 PM
To: "dev@zest.apache.org" <dev@zest.apache.org>
Subject: Re: GeoSpatial Query support

>Hi Adam & Chris,
>
>some notes on the Zest spatial implementations. It is in general
>structured
>in three main components :
>
>1.) Zest core that defines spatial values, like TPoint, TPolygon, ... as
>well as the query DSL that allows to define spatial queries for the
>underlying indexing system. The values are derived from the
>http://geojson.org/ spec. The Zest spatial core is dependency free, e.g.
>no
>3th party libraries are used.
>
>2.) The indexing system that is capable to i.) index spatial values and
>ii.)  "executes" the spatial queries. For now we are supporting
>ElasticSearch for spatial queries. Means we are delegating the query to ES
>while "traversing" the ES native query with some required metadata. There
>are a number of "nasty" things while using ElasticSearch for spatial
>queries like the "mappings", where one has to declare spatial types during
>indexing, define how to index values (as Points or as Shapes) but this
>is encapsulated by the Zest ES spatial extension. Also one has to deal
>with
>projections, as ES just supports WGS84. Means when your query is using a
>different projection, then we have to convert it to WGS84 while consider
>the conversion accuracy..
>
>3.) Supporting library that contains the conversion and projection code -
>we are using here the spatial4j library.
>
>Sample to create a spatial value :
>
>       TPolygon polygon = TPolygon(module)
>                .shell
>                        (
>                                new double[][]
>                                        {
>                                                {0, 0},
>                                                {3, 0},
>                                                {1, 3},
>                                                {0, 3},
>                                                {0, 0}
>                                        }
>                        )
>                .withHoles(
>                        TLinearRing(module).ring(new double[][]
>                                {
>                                        {0, 0},
>                                        {1, 0},
>                                        {1, 1},
>                                        {0, 1},
>                                        {0, 0}
>
>                                }).geometry()
>                        , TLinearRing(module).ring(new double[][]
>                                {
>                                        {0, 0},
>                                        {2, 0},
>                                        {2, 2},
>                                        {0, 2},
>                                        {0, 0}
>                                }).geometry()
>                )
>                .geometry();
>
>Sample spatial query :
>
>        Query<A> query = unitOfWork.newQuery(
>                qb
>                        .where(and(
>                                        *ST_Disjoint*
>                                                (
>
>templateFor(A.class).point(),
>
>TPoint(module).y(2389280.7514562616).x(1286436.5975464052).geometry("
>*EPSG:27572*"),
>                                                        10, TUnit.METER
>                                                ),
>                                        *ST_Within*
>                                                (
>
>templateFor(A.class).point(),
>
>TPoint(module).y(48.13905780942574).x(11.57958984375).*geometry(*),
>                  
>100,TUnit.KILOMETER
>                                                )
>                                )
>                        ));
>        query.find();
>
>Here we are mixing two different projections in one query. Default
>projection is defined to be "EPSG:4326".
>ES is using geohashes or quadtree approach to index spatial values, for
>now
>we are using the quadtree approach. The required precision level
>(accuracy)
>can be defined
>in a configuration file :
>
>spatial = { \
>  \
>  Enabled : true, \
>  \
>  Indexer=\
>   { \
>    Method=\
>      { \
>        Type      = "GEO_POINT", \ <-- ES spatial indexing method
>(GEO_POINT or GEO_SHAPE)
>        Precision = 1m, \  * <-- Indexing accuracy aka quadtree levels*
>      }, \
>    Projection=\
>      { \
>*        ConversionEnabled  = true, \*
>*        ConversionAccuracy = 2m, \ <-- projection conversion accuracy
>while indexing*
>      }, \
>   }, \
>  \
>  \
>  Finder=\
>    { \
>    Projection=\
>      { \
>        ConversionEnabled  = true, \
>        ConversionAccuracy = 2m, \ *<-- projection conversion accuracy
>during query execution*
>      } \
>    } \
>  }
>
>Overall status:
>
>Ok, the Zest Spatial stuff is for now very simple. We support the
>following
>query expressions :
>
>- ST_Within
>- ST_Disjoint
>- ST_Intersects
>
>Due to implementation difficulties on ES I;m using a kind of "supported"
>matrix that declares what combinations are allowed :
>
>          // ST_Within
>        supports(enable(ST_WithinSpecification.class),
>propertyOf(AnyGeometry), filterOf(TPoint.class, TPolygon.class),
>enable(OrderBy), SpatialConfiguration.INDEXING_METHOD.GEO_POINT);
>        supports(enable(ST_WithinSpecification.class),
>propertyOf(AnyGeometry), filterOf(TPoint.class, TPolygon.class),
>disable(OrderBy), SpatialConfiguration.INDEXING_METHOD.GEO_SHAPE);
>
>        // ST_Disjoint
>        supports(disable(ST_DisjointSpecification.class),
>propertyOf(AnyGeometry), filterOf(TPoint.class, TPolygon.class),
>enable(OrderBy), SpatialConfiguration.INDEXING_METHOD.GEO_POINT);
>        supports(enable(ST_DisjointSpecification.class),
>propertyOf(AnyGeometry), filterOf(TPoint.class, TPolygon.class),
>disable(OrderBy), SpatialConfiguration.INDEXING_METHOD.GEO_SHAPE);
>
>
>        // ST_Intersects
>        supports(disable(ST_IntersectsSpecification.class),
>propertyOf(AnyGeometry), filterOf(TPoint.class, TPolygon.class),
>enable(OrderBy), SpatialConfiguration.INDEXING_METHOD.GEO_POINT);
>        supports(enable(ST_IntersectsSpecification.class),
>propertyOf(AnyGeometry), filterOf(TPoint.class, TPolygon.class),
>disable(OrderBy), SpatialConfiguration.INDEXING_METHOD.GEO_SHAPE);
>
>Sample :
>
>        supports(enable(ST_WithinSpecification.class),
>propertyOf(AnyGeometry), filterOf(TPoint.class, TPolygon.class),
>disable(OrderBy), SpatialConfiguration.INDEXING_METHOD.GEO_SHAPE);
>
>== Enable support for *ST_With *on *AnyGeometry* and allow *TPoint*,
>*TPolygon* as filter and *disable ordering* when *GEO_SHAPING* indexing
>method is used.
>
>This is indeed not a perfect solution, but using that approach we are able
>to express the supported constellations and are able to generate a
>"speaking" exception when
>a unsupported query is defined.
>
>Stability :
>
>I can not say a lot on long-term stability yet. But I take the
>OpenStreetMap Planet.osm and "pump" it to Apache ZEST and did some
>concurrent queries on it
>without any serious issues. In general ES performs really great on spatial
>data !
>
>Cheers,
>Jiri
>
>
>
>
>
>
>
>
>2015-06-03 17:12 GMT+02:00 <estrada.adam@gmail.com>:
>
>> Most excellent, Dr. Mattmann! @sis, we should definitely collaborate
>>with
>> @zest on this one. If they are implementing simple query functionality,
>> maybe they can take advantage of the work Marc has already done and also
>> use the projection support Martin has been committing.
>>
>> Thoughts?
>> Adam
>>
>> Sent from my iPhone
>>
>> > On Jun 3, 2015, at 11:00 AM, Mattmann, Chris A (3980) <
>> chris.a.mattmann@jpl.nasa.gov> wrote:
>> >
>> > Maybe this is a good integration point for sis.apache.org?
>> >
>> > CC’ing the Apache SIS list for visibility
>> >
>> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> > Chris Mattmann, Ph.D.
>> > Chief Architect
>> > Instrument Software and Science Data Systems Section (398)
>> > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> > Office: 168-519, Mailstop: 168-527
>> > Email: chris.a.mattmann@nasa.gov
>> > WWW:  http://sunset.usc.edu/~mattmann/
>> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> > Adjunct Associate Professor, Computer Science Department
>> > University of Southern California, Los Angeles, CA 90089 USA
>> > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> >
>> >
>> >
>> >
>> > -----Original Message-----
>> > From: Jiri Jetmar <juergen.jetmar@gmail.com>
>> > Reply-To: "dev@zest.apache.org" <dev@zest.apache.org>
>> > Date: Wednesday, June 3, 2015 at 7:45 AM
>> > To: "dev@zest.apache.org" <dev@zest.apache.org>
>> > Subject: Re: GeoSpatial Query support
>> >
>> >> Hi Niclas,
>> >>
>> >> this ST_XX follows the http://en.wikipedia.org/wiki/DE-9IM and
>> >> http://en.wikipedia.org/wiki/Simple_Features standards that are
>>indeed
>> >> "SQL" related. I used this expression to be similar with e.g.
>>PostGIS,
>> >> means when someone knows how to use spatial expressions on PostGIS,
>> >> he should be able to understand the Apache Zest approach.
>> >>
>> >> But clearly I can "lowercase" the ST_Xx expressions in the code.
>> >>
>> >> Cheers,
>> >> jj
>> >>
>> >> 2015-06-03 16:22 GMT+02:00 Niclas Hedhman <niclas@hedhman.org>:
>> >>
>> >>> Jiri,
>> >>> There is one thing that bothers me; ST_Within and similar doesn't
>> follow
>> >>> the Java convention at all. I have researched and seen that many DBs
>> >>> uses
>> >>> this, but don't you think we should lower case st_ instead for
>>methods?
>> >>>
>> >>>
>> >>> Cheers
>> >>> --
>> >>> Niclas Hedhman, Software Developer
>> >>> http://zest.apache.org - New Energy for Java
>> >
>>

Mime
View raw message