asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Riyafa Abdul Hameed <riyafa...@cse.mrt.ac.lk>
Subject Re: ASTERIXDB-1371: Support the standard GIS objects
Date Sat, 03 Jun 2017 14:27:00 GMT
Hi,

So far I have been trying out Esri geometry api[1]. I parsed a csv file to
read data in WKT format, converted it to OGCGeometry objects and calculated
the average areas of the objects. Further I converted this OGCGeometry
objects to WKB format and stored in a file. Then I read back the file and
calculated the average areas of these objects. Thus I have learned the use
of the api.
Also since the project would focus on TDD(Test Driven Development) I have
been focusing on writing AQL queries on calculating the average areas,
Range query, k nearest neighbor and spatial joins using both the current
support of spatial objects and future support of the spatial objects. These
queries are to be used as integrations tests in the future.

[1] https://github.com/Esri/geometry-api-java

Thank you.
Yours sincerely,
Riyafa

On 17 May 2017 at 23:35, Mike Carey <dtabass@gmail.com> wrote:

> Sounds like a plan!  :-)
>
>
> On 5/17/17 7:25 AM, Riyafa Abdul Hameed wrote:
>
>> Hi,
>>
>> Through a discussion that I had with Ahmed, I have been able to clarify
>> the
>> following points:
>>
>>
>>     - A new data type which is known as Geometry would be supported
>>     - Esri library[1] shall be used for parsing and operating on WKT and
>>     GeoJSON formats
>>     - Functions such as parseWKT and parseGeoJSON would be used for
>> parsing
>>     text in these formats to Geometry datatype
>>     - Internally Geometry datatype would be represented in WKB(Well Known
>>     Binary) format
>>     - Functions shall be written(if time permits) to convert currently
>>     supported datatypes such as point, polygon, circle, and rectangle to
>>     Geometry datatype
>>
>> [1]
>> https://github.com/Esri/geometry-api-java/blob/master/src/
>> main/java/com/esri/core/geometry/Geometry.java
>>
>> Thank you.
>>
>> Yours sincerely,
>>
>> Riyafa
>>
>> On 15 May 2017 at 20:13, Riyafa Abdul Hameed <riyafa@apache.org> wrote:
>>
>> Hi,
>>>
>>> In a different thread Preston had asked about the format of the
>>> datatypes.
>>> As I gather from this discussion current format supported in AsterixDB is
>>> different from WKT or GeoJSON. So the tasks in resolving this issue, does
>>> it involve supporting the new Geo objects in the *current format* and
>>> then in GeoJSON and/or WKT or is it about supporting all the GIS objects
>>> in
>>> GeoJSON and/or WKT *only*?
>>> (I have already started writing code to support triangle so as to get the
>>> hang of implementation details)
>>>
>>> Thank you.
>>>
>>> Yours sincerely,
>>> Riyafa
>>>
>>> On 14 May 2017 at 09:36, Riyafa Abdul Hameed <riyafa@apache.org> wrote:
>>>
>>> Hi,
>>>>
>>>> I have been exploring the current implementations of point, polygon,
>>>> circle, and rectangle. These have been implemented as Primitive Types
>>>> <https://ci.apache.org/projects/asterixdb/datamodel.html#PrimitiveTypes
>>>> >
>>>> and like Ahmed mentioned they have have an internal
>>>> representation in WKB. The data types that require to be implemented
>>>> are:
>>>>
>>>>     - MultiPoint
>>>>     - MultiLineString
>>>>     - MultiPolygon
>>>>     - Triangle
>>>>     - CircularString
>>>>     - Curve
>>>>     - MultiCurve
>>>>     - CompoundCurve
>>>>     - CurvePolygon
>>>>     - Surface
>>>>     - MultiSurface
>>>>     - PolyhedralSurface
>>>>     - TIN (Triangulated irregular network)
>>>>     - GeometryCollection
>>>>
>>>>
>>>> As I understand by looking at the implementation of point and poygon
>>>> they
>>>> are implemented as functions and builtintypes in AsterixDB internally.
>>>> So I
>>>> thought of initially implementing these new types internally. I thought
>>>> of
>>>> starting with Triangle and once I get the hang of it by having the
>>>> implementation reviewed I can continue with the implementation of the
>>>> remaining types. As for using Esri API, since it is not currently used I
>>>> thought it can come handy when implementing the required operations on
>>>> each
>>>> of these data types.
>>>>
>>>> Thank you.
>>>>
>>>> Yours sincerely,
>>>>
>>>> Riyafa
>>>>
>>>>
>>>>
>>>>
>>>> On 14 May 2017 at 02:56, Ahmed Eldawy <eldawy@cs.ucr.edu> wrote:
>>>>
>>>> The notice of the two formats is important. The GeoJSON (or WKT) format
>>>>> is
>>>>> only for importing text files. AsterixDB should have an internal
>>>>> representation for geometries. The only standard I know in this matter
>>>>> is
>>>>> Well-known Binary (WKB) which is also supported by Esri API and JTS.
>>>>> @Wail: AsterixDB is not supposed to automatically detect that a field
>>>>> is
>>>>> GeoJSON. The user should provide this information while importing a
>>>>> file,
>>>>> probably, by defining the schema with a Geometry field.
>>>>>
>>>>> Thanks
>>>>> Ahmed
>>>>>
>>>>> On Fri, May 12, 2017 at 2:31 PM, Wail Alkowaileet <wael.y.k@gmail.com>
>>>>> wrote:
>>>>>
>>>>> One small thing about GeoJSON ...
>>>>>>
>>>>>>  From the JSON/ADM parser perspective, GeoJSON is still JSON.
>>>>>> "Disambiguating" or distinguishing between them might not be a
>>>>>> straightforward process. Getting GeoJSON parsed into AsterixDB
>>>>>> internal
>>>>>> geometries would probably be the first step...
>>>>>>
>>>>>> On Fri, May 12, 2017 at 11:15 PM, Mike Carey <dtabass@gmail.com>
>>>>>>
>>>>> wrote:
>>>>>
>>>>>> Just to chime in briefly on the "format" thread - there are two
>>>>>>>
>>>>>> formats
>>>>>
>>>>>> to
>>>>>>
>>>>>>> keep in mind - the input format (serialized format, e.g., how
JSON
>>>>>>>
>>>>>> relates
>>>>>>
>>>>>>> to the spatial types) and the internal format (which in AsterixDB
is
>>>>>>>
>>>>>> a
>>>>>
>>>>>> different, more efficient, binary format).  We can look at both in
>>>>>>>
>>>>>> the
>>>>>
>>>>>> project. Also important are the OPERATIONS that go with the format
>>>>>>>
>>>>>> (i.e.,
>>>>>
>>>>>> the functions that we'll have in the query language for operating
on,
>>>>>>> writing predicates about, etc., the spatial data).
>>>>>>>
>>>>>>> Cheers,
>>>>>>>
>>>>>>> Mike
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On 5/11/17 2:28 PM, Ahmed Eldawy wrote:
>>>>>>>
>>>>>>> Hi Riyafa,
>>>>>>>>
>>>>>>>> I'm glad you started looking into the details. I agree with
Mike
>>>>>>>>
>>>>>>> that
>>>>>
>>>>>> you
>>>>>>
>>>>>>> need to study the current geo support first. It will be highly
>>>>>>>>
>>>>>>> desirable
>>>>>
>>>>>> for your work to be compatible with the current support so that we
>>>>>>>>
>>>>>>> can
>>>>>
>>>>>> seamlessly unify the underlying code without disrupting the
>>>>>>>>
>>>>>>> high-level
>>>>>
>>>>>> API.
>>>>>>>> As for the format, it would be nice to support both WKT and
GeoJSON
>>>>>>>>
>>>>>>> as
>>>>>
>>>>>> they
>>>>>>>> are both widely used. However, I think we should start with
GeoJSON
>>>>>>>>
>>>>>>> since
>>>>>>
>>>>>>> it is becoming more popular with modern devices, e.g., GPS, smart
>>>>>>>>
>>>>>>> phones,
>>>>>>
>>>>>>> and IoT sensors. Later, we can support WKT as well. It will be
a
>>>>>>>>
>>>>>>> matter
>>>>>
>>>>>> of
>>>>>>
>>>>>>> writing a different parse function.
>>>>>>>> Esri library [https://github.com/Esri/geometry-api-java]
supports
>>>>>>>>
>>>>>>> both
>>>>>
>>>>>> WKT
>>>>>>>> and GeoJSON. You can study it and see if we can use it in
our
>>>>>>>>
>>>>>>> project.
>>>>>
>>>>>> Thanks
>>>>>>>> Ahmed
>>>>>>>>
>>>>>>>> On Wed, May 10, 2017 at 7:11 AM, Mike Carey <dtabass@gmail.com>
>>>>>>>>
>>>>>>> wrote:
>>>>>
>>>>>> I will leave it to the official GSC mentor (who's also a leading
>>>>>>>>
>>>>>>> expert
>>>>>
>>>>>> on
>>>>>>
>>>>>>> big spatial data) to direct - I was just suggesting that step
0
>>>>>>>>>
>>>>>>>> should
>>>>>
>>>>>> be
>>>>>>
>>>>>>> to become familiar with what's already there currently, to have
a
>>>>>>>>>
>>>>>>>> working
>>>>>>
>>>>>>> knowledge of that as background.
>>>>>>>>>
>>>>>>>>> :-)
>>>>>>>>>
>>>>>>>>> Looking forward to seeing this project unfold!
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>>
>>>>>>>>> Mike
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On 5/9/17 10:14 PM, Riyafa Abdul Hameed wrote:
>>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>>> As I understand by playing with current support of
GIS objects(
>>>>>>>>>>
>>>>>>>>> point,
>>>>>
>>>>>> polygon, circle, and rectangle) is similar to the Well known text
>>>>>>>>>> format--correct me if I am mistaken. Hence initially
we could
>>>>>>>>>>
>>>>>>>>> support
>>>>>
>>>>>> other
>>>>>>>>>> GIS objects in WKT and support GeoJSON if time permits.
>>>>>>>>>>
>>>>>>>>>> Thank you.
>>>>>>>>>> Yours sincerely,
>>>>>>>>>> Riyafa
>>>>>>>>>>
>>>>>>>>>> On 8 May 2017 at 23:31, Mike Carey <dtabass@gmail.com>
wrote:
>>>>>>>>>>
>>>>>>>>>> I would also suggest playing with the current geo
support in
>>>>>>>>>>
>>>>>>>>> AsterixDB
>>>>>
>>>>>> (curretn types and indexing and functions in queries) to get
>>>>>>>>>>>
>>>>>>>>>> warmed
>>>>>
>>>>>> up.
>>>>>>
>>>>>>> Welcome aboard...!!
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>>
>>>>>>>>>>> Mike
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On 5/8/17 8:51 AM, Riyafa Abdul Hameed wrote:
>>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>>
>>>>>>>>>>> I have been selected to contribute to the issue
ASTERIXDB-1371
>>>>>>>>>>>> <https://issues.apache.org/jira/browse/ASTERIXDB-1371>
for GSoC
>>>>>>>>>>>>
>>>>>>>>>>> this
>>>>>>
>>>>>>> time.
>>>>>>>>>>>> This being the community bonding period I
am trying to
>>>>>>>>>>>>
>>>>>>>>>>> familiarize
>>>>>
>>>>>> myself
>>>>>>>>>>>> with the code base of AsterixDB and to get
a grasp of the task.
>>>>>>>>>>>>
>>>>>>>>>>>> I am under the impression that the package
*
>>>>>>>>>>>>
>>>>>>>>>>> org.apache.asterix.om
>>>>>
>>>>>> <http://org.apache.asterix.om> *has the classes for handling
>>>>>>>>>>>>
>>>>>>>>>>> data
>>>>>
>>>>>> models
>>>>>>>>>>>> for AsterixDB and have been looking into
them to figure out the
>>>>>>>>>>>> implementation details. Please correct me
if I am mistaken.
>>>>>>>>>>>>
>>>>>>>>>>>> I have also been reading on the specification
for well known
>>>>>>>>>>>>
>>>>>>>>>>> text[1]
>>>>>
>>>>>> and
>>>>>>>>>>>> GeoJSON[2] and have been trying to figure
out if implementing
>>>>>>>>>>>>
>>>>>>>>>>> one of
>>>>>
>>>>>> them
>>>>>>>>>>>> would suffice (if so which one) or if both
needs to be
>>>>>>>>>>>>
>>>>>>>>>>> implemented.
>>>>>
>>>>>> If
>>>>>>
>>>>>>> both
>>>>>>>>>>>> needs to be implemented we should decide
which needs to be
>>>>>>>>>>>>
>>>>>>>>>>> implemented
>>>>>>
>>>>>>> first. I was thinking of going for GeoJSON as it seems to have
a
>>>>>>>>>>>>
>>>>>>>>>>> wider
>>>>>>
>>>>>>> usage.
>>>>>>>>>>>>
>>>>>>>>>>>> Any suggestions on how I should proceed with
the project would
>>>>>>>>>>>>
>>>>>>>>>>> be
>>>>>
>>>>>> highly
>>>>>>>>>>>> valued.
>>>>>>>>>>>>
>>>>>>>>>>>> [1] http://docs.opengeospatial.org/is/12-063r5/12-063r5.html
>>>>>>>>>>>> [2] https://tools.ietf.org/html/rfc7946
>>>>>>>>>>>>
>>>>>>>>>>>> Thank you.
>>>>>>>>>>>>
>>>>>>>>>>>> Yours sincerely,
>>>>>>>>>>>> Riyafa
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>> --
>>>>>>
>>>>>> *Regards,*
>>>>>> Wail Alkowaileet
>>>>>>
>>>>>>
>>>>
>


-- 
Riyafa Abdul Hameed
Undergraduate, University of Moratuwa

Email: riyafa.12@cse.mrt.ac.lk
Website: https://riyafa.wordpress.com/ <http://riyafa.wordpress.com/>
<http://facebook.com/riyafa.ahf>  <http://lk.linkedin.com/in/riyafa>
<http://twitter.com/Riyafa1>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message