asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Riyafa Abdul Hameed <riy...@apache.org>
Subject Re: Parse GeoJSON data into a record in AsterixDB
Date Thu, 22 Jun 2017 14:58:49 GMT
Dear all,

Thank you very much. I hadn't thought of an AnyObject type. Now I am able
to parse GeoJSON using the following:

DROP DATAVERSE GeoData IF EXISTS;
CREATE DATAVERSE GeoData;
 USE GeoData;
CREATE TYPE AnyObject AS {};
CREATE TYPE FeatureType AS {
         id: UUID,
         `type`: string,
        `geometry`: AnyObject,
        properties: AnyObject
};
CREATE DATASET Geometries (FeatureType) PRIMARY KEY id autogenerated;
INSERT INTO Geometries ([
{

   "type":"Feature",
   "geometry":{
      "type":"Point",
      "coordinates":[
         -118.40,
         33.93
      ]
   },
   "properties":{
      "code":"LAX",
      "elevation":38
   }
}
]);
SELECT * FROM Geometries;

Thanks again.
Riyafa

On 22 June 2017 at 02:37, Ahmed Eldawy <eldawy@cs.ucr.edu> wrote:

> Hi Riyafa,
>
> I think you should use the terms *feature* and *geometry* to avoid
> confusion like the following example.
>
> CREATE TYPE AnyObject AS {};
>
> CREATE TYPE *FeatureType* AS {
>          `type`: string,
>          the_geom: *geometry*,
>          properties: AnyObject
> };
>
> The internal 'geometry' attribute is what holds the shape geometry while
> the outer FeatureType associates additional attributes and properties to
> that geometry.
>
> As we discussed in our last call, the first step is to parse GeoJSON as a
> regular JSON file and then parse the geometry using a UDF. In this case,
> you might have something like:
>
> CREATE TYPE *FeatureType* AS {
>          `type`: string,
>          the_geom: *AnyObject*,
>          properties: AnyObject
> };
>
> This will allow you to parse the file without modifying the existing JSON
> parser. Then, you can define a UDF called "ParseGeoJSON" which takes as
> input the "the_geom" attribute and returns a parsed geometry attribute.
> The next step should avoid this additional step and should automatically
> detect and parse the geometry attribute directly from GeoJJSON.
>
>
> Thanks
> Ahmed
>
> On Wed, Jun 21, 2017 at 11:40 AM, Yingyi Bu <buyingyi@gmail.com> wrote:
>
>> >> type appears to be keyword
>> `type` would make it valid.
>>
>> >> We can't use the defining type within the same type recursively (ie.
>> GeometryType within GeometryType)
>>
>> We don't support recursive type definition.
>>
>> >> The type object cannot be resolved
>>
>>  We don't have a builtin name for a completely open type, but you can
>> define one.
>>
>>
>> What you can do is:
>>
>> CREATE TYPE AnyObject AS {};
>>
>> CREATE TYPE GeometryType AS {
>>          `type`: string,
>>          geometry: SomeType;
>>          properties: AnyObject
>> }
>>
>>
>> Best,
>> Yingyi
>>
>> On Wed, Jun 21, 2017 at 6:33 AM, Mike Carey <dtabass@gmail.com> wrote:
>>
>> > One approach would be to be silent about properties - and then it could
>> be
>> > there anyway - however, that wouldn't allow you to state the requirement
>> > (?) that it must be called properties and/or that it must be an object
>> (not
>> > a scalar).  That could work for now, perhaps?  We need to have an "any
>> > record type" type name - we've noted a desire for that - unfortunately
>> we
>> > don't have one I don't think.  I believe the concept is there inside the
>> > code, in the type-related areas, but we don't have a keyword like name
>> for
>> > it.  (@Yingyi - comments?)  And we do also have a restriction (at the
>> type
>> > level) that precludes recursion (regular or mutual) in type
>> definitions; we
>> > probably need to do something about that someday as well.
>> >
>> > In the meantime, these things could be handled (weakly) by documenting
>> > what's expected/allowed in this setting.
>> >
>> > Cheers,
>> >
>> > Mike
>> >
>> > PS - I wonder if JSON Schema has the expressiveness for this?
>> >
>> > On 6/21/17 2:26 AM, Riyafa Abdul Hameed wrote:
>> >
>> > Hi,
>> >
>> > I would like to parse the following or any GeoJSON type[1] to a record
>> in
>> > AsterixDB:
>> > {
>> >    "type":"Feature",
>> >    "geometry":{
>> >       "type":"Point",
>> >       "coordinates":[
>> >          -118.40,
>> >          33.93
>> >       ]
>> >    },
>> >    "properties":{
>> >       "code":"LAX",
>> >       "elevation":38
>> >    }
>> > }
>> >
>> > The value of properties is optional and is a variable that is it can be
>> > any type of object. What is the most suitable datatype to use to
>> represent
>> > properties in this case?
>> > Is something like the following possible?
>> >
>> > CREATE TYPE GeometryType AS {
>> >      type: string,
>> >      geometry: GeometryType,
>> >      properties: object
>> > };
>> >
>> > I came up with the above because there's a derived type called
>> objects[2]
>> > in AsterixDB. The above doesn't work because of the following reasons:
>> >
>> >    - type appears to be keyword
>> >    - We can't use the defining type within the same type recursively
>> (ie.
>> >    GeometryType within GeometryType)
>> >    - The type object cannot be resolved
>> >
>> > Any suggestions on how a GeoJSON can be parsed into AsterixDB?
>> >
>> > [1] https://tools.ietf.org/html/rfc7946
>> > [2] https://ci.apache.org/projects/asterixdb/datamodel.html#Deri
>> > vedTypesObject
>> >
>> > Thank you
>> > Yours sincerely,
>> > Riyafa
>> >
>> >
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message