asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Carey <dtab...@gmail.com>
Subject Re: ASTERIXDB-1371: Support the standard GIS objects
Date Sun, 04 Jun 2017 23:14:36 GMT
Cool!  (BTW, the preferred AsterixDB QL is SQL++ nowadays - AQL is being 
deprecated - the same functionality is available in both QL's in terms 
of the spatial stuff.)


On 6/3/17 7:27 AM, Riyafa Abdul Hameed wrote:
> Hi,
>
> So far I have been trying out Esri geometry api[1]. I parsed a csv file to
> read data in WKT format, converted it to OGCGeometry objects and calculated
> the average areas of the objects. Further I converted this OGCGeometry
> objects to WKB format and stored in a file. Then I read back the file and
> calculated the average areas of these objects. Thus I have learned the use
> of the api.
> Also since the project would focus on TDD(Test Driven Development) I have
> been focusing on writing AQL queries on calculating the average areas,
> Range query, k nearest neighbor and spatial joins using both the current
> support of spatial objects and future support of the spatial objects. These
> queries are to be used as integrations tests in the future.
>
> [1] https://github.com/Esri/geometry-api-java
>
> Thank you.
> Yours sincerely,
> Riyafa
>
> On 17 May 2017 at 23:35, Mike Carey <dtabass@gmail.com> wrote:
>
>> Sounds like a plan!  :-)
>>
>>
>> On 5/17/17 7:25 AM, Riyafa Abdul Hameed wrote:
>>
>>> Hi,
>>>
>>> Through a discussion that I had with Ahmed, I have been able to clarify
>>> the
>>> following points:
>>>
>>>
>>>      - A new data type which is known as Geometry would be supported
>>>      - Esri library[1] shall be used for parsing and operating on WKT and
>>>      GeoJSON formats
>>>      - Functions such as parseWKT and parseGeoJSON would be used for
>>> parsing
>>>      text in these formats to Geometry datatype
>>>      - Internally Geometry datatype would be represented in WKB(Well Known
>>>      Binary) format
>>>      - Functions shall be written(if time permits) to convert currently
>>>      supported datatypes such as point, polygon, circle, and rectangle to
>>>      Geometry datatype
>>>
>>> [1]
>>> https://github.com/Esri/geometry-api-java/blob/master/src/
>>> main/java/com/esri/core/geometry/Geometry.java
>>>
>>> Thank you.
>>>
>>> Yours sincerely,
>>>
>>> Riyafa
>>>
>>> On 15 May 2017 at 20:13, Riyafa Abdul Hameed <riyafa@apache.org> wrote:
>>>
>>> Hi,
>>>> In a different thread Preston had asked about the format of the
>>>> datatypes.
>>>> As I gather from this discussion current format supported in AsterixDB is
>>>> different from WKT or GeoJSON. So the tasks in resolving this issue, does
>>>> it involve supporting the new Geo objects in the *current format* and
>>>> then in GeoJSON and/or WKT or is it about supporting all the GIS objects
>>>> in
>>>> GeoJSON and/or WKT *only*?
>>>> (I have already started writing code to support triangle so as to get the
>>>> hang of implementation details)
>>>>
>>>> Thank you.
>>>>
>>>> Yours sincerely,
>>>> Riyafa
>>>>
>>>> On 14 May 2017 at 09:36, Riyafa Abdul Hameed <riyafa@apache.org> wrote:
>>>>
>>>> Hi,
>>>>> I have been exploring the current implementations of point, polygon,
>>>>> circle, and rectangle. These have been implemented as Primitive Types
>>>>> <https://ci.apache.org/projects/asterixdb/datamodel.html#PrimitiveTypes
>>>>> and like Ahmed mentioned they have have an internal
>>>>> representation in WKB. The data types that require to be implemented
>>>>> are:
>>>>>
>>>>>      - MultiPoint
>>>>>      - MultiLineString
>>>>>      - MultiPolygon
>>>>>      - Triangle
>>>>>      - CircularString
>>>>>      - Curve
>>>>>      - MultiCurve
>>>>>      - CompoundCurve
>>>>>      - CurvePolygon
>>>>>      - Surface
>>>>>      - MultiSurface
>>>>>      - PolyhedralSurface
>>>>>      - TIN (Triangulated irregular network)
>>>>>      - GeometryCollection
>>>>>
>>>>>
>>>>> As I understand by looking at the implementation of point and poygon
>>>>> they
>>>>> are implemented as functions and builtintypes in AsterixDB internally.
>>>>> So I
>>>>> thought of initially implementing these new types internally. I thought
>>>>> of
>>>>> starting with Triangle and once I get the hang of it by having the
>>>>> implementation reviewed I can continue with the implementation of the
>>>>> remaining types. As for using Esri API, since it is not currently used
I
>>>>> thought it can come handy when implementing the required operations on
>>>>> each
>>>>> of these data types.
>>>>>
>>>>> Thank you.
>>>>>
>>>>> Yours sincerely,
>>>>>
>>>>> Riyafa
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On 14 May 2017 at 02:56, Ahmed Eldawy <eldawy@cs.ucr.edu> wrote:
>>>>>
>>>>> The notice of the two formats is important. The GeoJSON (or WKT) format
>>>>>> is
>>>>>> only for importing text files. AsterixDB should have an internal
>>>>>> representation for geometries. The only standard I know in this matter
>>>>>> is
>>>>>> Well-known Binary (WKB) which is also supported by Esri API and JTS.
>>>>>> @Wail: AsterixDB is not supposed to automatically detect that a field
>>>>>> is
>>>>>> GeoJSON. The user should provide this information while importing
a
>>>>>> file,
>>>>>> probably, by defining the schema with a Geometry field.
>>>>>>
>>>>>> Thanks
>>>>>> Ahmed
>>>>>>
>>>>>> On Fri, May 12, 2017 at 2:31 PM, Wail Alkowaileet <wael.y.k@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>> One small thing about GeoJSON ...
>>>>>>>   From the JSON/ADM parser perspective, GeoJSON is still JSON.
>>>>>>> "Disambiguating" or distinguishing between them might not be
a
>>>>>>> straightforward process. Getting GeoJSON parsed into AsterixDB
>>>>>>> internal
>>>>>>> geometries would probably be the first step...
>>>>>>>
>>>>>>> On Fri, May 12, 2017 at 11:15 PM, Mike Carey <dtabass@gmail.com>
>>>>>>>
>>>>>> wrote:
>>>>>>
>>>>>>> Just to chime in briefly on the "format" thread - there are two
>>>>>>> formats
>>>>>>> to
>>>>>>>
>>>>>>>> keep in mind - the input format (serialized format, e.g.,
how JSON
>>>>>>>>
>>>>>>> relates
>>>>>>>
>>>>>>>> to the spatial types) and the internal format (which in AsterixDB
is
>>>>>>>>
>>>>>>> a
>>>>>>> different, more efficient, binary format).  We can look at both
in
>>>>>>> the
>>>>>>> project. Also important are the OPERATIONS that go with the format
>>>>>>> (i.e.,
>>>>>>> the functions that we'll have in the query language for operating
on,
>>>>>>>> writing predicates about, etc., the spatial data).
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>>
>>>>>>>> Mike
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 5/11/17 2:28 PM, Ahmed Eldawy wrote:
>>>>>>>>
>>>>>>>> Hi Riyafa,
>>>>>>>>> I'm glad you started looking into the details. I agree
with Mike
>>>>>>>>>
>>>>>>>> that
>>>>>>> you
>>>>>>>
>>>>>>>> need to study the current geo support first. It will be highly
>>>>>>>> desirable
>>>>>>> for your work to be compatible with the current support so that
we
>>>>>>>> can
>>>>>>> seamlessly unify the underlying code without disrupting the
>>>>>>>> high-level
>>>>>>> API.
>>>>>>>>> As for the format, it would be nice to support both WKT
and GeoJSON
>>>>>>>>>
>>>>>>>> as
>>>>>>> they
>>>>>>>>> are both widely used. However, I think we should start
with GeoJSON
>>>>>>>>>
>>>>>>>> since
>>>>>>>> it is becoming more popular with modern devices, e.g., GPS,
smart
>>>>>>>> phones,
>>>>>>>> and IoT sensors. Later, we can support WKT as well. It will
be a
>>>>>>>> matter
>>>>>>> of
>>>>>>>
>>>>>>>> writing a different parse function.
>>>>>>>>> Esri library [https://github.com/Esri/geometry-api-java]
supports
>>>>>>>>>
>>>>>>>> both
>>>>>>> WKT
>>>>>>>>> and GeoJSON. You can study it and see if we can use it
in our
>>>>>>>>>
>>>>>>>> project.
>>>>>>> Thanks
>>>>>>>>> Ahmed
>>>>>>>>>
>>>>>>>>> On Wed, May 10, 2017 at 7:11 AM, Mike Carey <dtabass@gmail.com>
>>>>>>>>>
>>>>>>>> wrote:
>>>>>>> I will leave it to the official GSC mentor (who's also a leading
>>>>>>>> expert
>>>>>>> on
>>>>>>>
>>>>>>>> big spatial data) to direct - I was just suggesting that
step 0
>>>>>>>>> should
>>>>>>> be
>>>>>>>
>>>>>>>> to become familiar with what's already there currently, to
have a
>>>>>>>>> working
>>>>>>>> knowledge of that as background.
>>>>>>>>>> :-)
>>>>>>>>>>
>>>>>>>>>> Looking forward to seeing this project unfold!
>>>>>>>>>>
>>>>>>>>>> Cheers,
>>>>>>>>>>
>>>>>>>>>> Mike
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On 5/9/17 10:14 PM, Riyafa Abdul Hameed wrote:
>>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>>> As I understand by playing with current support
of GIS objects(
>>>>>>>>>>>
>>>>>>>>>> point,
>>>>>>> polygon, circle, and rectangle) is similar to the Well known
text
>>>>>>>>>>> format--correct me if I am mistaken. Hence initially
we could
>>>>>>>>>>>
>>>>>>>>>> support
>>>>>>> other
>>>>>>>>>>> GIS objects in WKT and support GeoJSON if time
permits.
>>>>>>>>>>>
>>>>>>>>>>> Thank you.
>>>>>>>>>>> Yours sincerely,
>>>>>>>>>>> Riyafa
>>>>>>>>>>>
>>>>>>>>>>> On 8 May 2017 at 23:31, Mike Carey <dtabass@gmail.com>
wrote:
>>>>>>>>>>>
>>>>>>>>>>> I would also suggest playing with the current
geo support in
>>>>>>>>>>>
>>>>>>>>>> AsterixDB
>>>>>>> (curretn types and indexing and functions in queries) to get
>>>>>>>>>>> warmed
>>>>>>> up.
>>>>>>>
>>>>>>>> Welcome aboard...!!
>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>
>>>>>>>>>>>> Mike
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On 5/8/17 8:51 AM, Riyafa Abdul Hameed wrote:
>>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I have been selected to contribute to the
issue ASTERIXDB-1371
>>>>>>>>>>>>> <https://issues.apache.org/jira/browse/ASTERIXDB-1371>
for GSoC
>>>>>>>>>>>>>
>>>>>>>>>>>> this
>>>>>>>> time.
>>>>>>>>>>>>> This being the community bonding period
I am trying to
>>>>>>>>>>>>>
>>>>>>>>>>>> familiarize
>>>>>>> myself
>>>>>>>>>>>>> with the code base of AsterixDB and to
get a grasp of the task.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I am under the impression that the package
*
>>>>>>>>>>>>>
>>>>>>>>>>>> org.apache.asterix.om
>>>>>>> <http://org.apache.asterix.om> *has the classes for handling
>>>>>>>>>>>> data
>>>>>>> models
>>>>>>>>>>>>> for AsterixDB and have been looking into
them to figure out the
>>>>>>>>>>>>> implementation details. Please correct
me if I am mistaken.
>>>>>>>>>>>>>
>>>>>>>>>>>>> I have also been reading on the specification
for well known
>>>>>>>>>>>>>
>>>>>>>>>>>> text[1]
>>>>>>> and
>>>>>>>>>>>>> GeoJSON[2] and have been trying to figure
out if implementing
>>>>>>>>>>>>>
>>>>>>>>>>>> one of
>>>>>>> them
>>>>>>>>>>>>> would suffice (if so which one) or if
both needs to be
>>>>>>>>>>>>>
>>>>>>>>>>>> implemented.
>>>>>>> If
>>>>>>>
>>>>>>>> both
>>>>>>>>>>>>> needs to be implemented we should decide
which needs to be
>>>>>>>>>>>>>
>>>>>>>>>>>> implemented
>>>>>>>> first. I was thinking of going for GeoJSON as it seems to
have a
>>>>>>>>>>>> wider
>>>>>>>> usage.
>>>>>>>>>>>>> Any suggestions on how I should proceed
with the project would
>>>>>>>>>>>>>
>>>>>>>>>>>> be
>>>>>>> highly
>>>>>>>>>>>>> valued.
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] http://docs.opengeospatial.org/is/12-063r5/12-063r5.html
>>>>>>>>>>>>> [2] https://tools.ietf.org/html/rfc7946
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thank you.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Yours sincerely,
>>>>>>>>>>>>> Riyafa
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>> --
>>>>>>>
>>>>>>> *Regards,*
>>>>>>> Wail Alkowaileet
>>>>>>>
>>>>>>>
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message