Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 15F7D200D69 for ; Wed, 27 Dec 2017 09:33:33 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 14584160C23; Wed, 27 Dec 2017 08:33:33 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id B0CC9160C20 for ; Wed, 27 Dec 2017 09:33:31 +0100 (CET) Received: (qmail 98435 invoked by uid 500); 27 Dec 2017 08:33:30 -0000 Mailing-List: contact user-help@geode.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@geode.apache.org Delivered-To: mailing list user@geode.apache.org Received: (qmail 98424 invoked by uid 99); 27 Dec 2017 08:33:30 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 27 Dec 2017 08:33:30 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 4827DC2C3E for ; Wed, 27 Dec 2017 08:33:30 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -0.421 X-Spam-Level: X-Spam-Status: No, score=-0.421 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Authentication-Results: spamd1-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=jpmorgan.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id kjVIfcw9eouZ for ; Wed, 27 Dec 2017 08:33:28 +0000 (UTC) Received: from sz14.jpmchase.com (sz14.jpmchase.com [159.53.78.162]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 96D945FB29 for ; Wed, 27 Dec 2017 08:33:27 +0000 (UTC) Received: from vsin21p1045.svr.us.jpmchase.net ([169.80.188.140]) by sz14.jpmchase.com with ESMTP id vBR8XJiu020596 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256(128 bits) verified NO); Wed, 27 Dec 2017 08:33:20 GMT DKIM-Filter: OpenDKIM Filter v2.7.5 sz14.jpmchase.com vBR8XJiu020596 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jpmorgan.com; s=d4815; t=1514363601; bh=hxQlBAreNuBfCrd5s6F2wt9nlCX2m7tL6+d4LRnASic=; h=From:To:Subject:Date; b=ZQFZz9gMEAbB17ZCOIlipQxlgVgWcIbqcKp1PxyUmA4yf4FOF5qWO/8r7dB9pQBud JPZDJTCzfag4RR+VFdZO71Jj00Eee9Yfim/FVBN9cgAtPDSoImsS2ircX4ACazQxt6 kkGHwjVuelLOd9W2qoYATEDcimNo7RR2myC5Lzg+bZ+hIbCb/SUkHozcPglGfW6NKS Sq0q0tTlDdvzHtq31Z43Zko/W5qVajnPNYkUMVMQGmMHTl729ixY/ber7HM/ir6wOL 6YKmxX8SOCLQExeQPWa9owRimrAr8AtIbIxoMrzpZJh9A4f9Jr210x1mIDwYUwUyH1 gtv+kePR01llg== Received: from smgh1.svr.us.jpmchase.net (smgh1.svr.us.jpmchase.net [169.124.133.102]) by se13.svr.us.jpmchase.net with ESMTP id vBR8XCkB031398 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384(256 bits) verified FAIL); Wed, 27 Dec 2017 08:33:19 GMT X-AuditID: a97c8566-ad3ff70000004041-27-5a435acdd2b4 Received: from SBEHHB002.exchad.jpmchase.net ( [169.69.59.58]) by smgh1.svr.us.jpmchase.net (Symantec Messaging Gateway) with SMTP id 0A.7B.16449.DCA534A5; Wed, 27 Dec 2017 08:33:18 +0000 (GMT) Received: from SPSCDI005.exchad.jpmchase.net (169.76.160.21) by SBEHHB002.exchad.jpmchase.net (169.83.4.250) with Microsoft SMTP Server (TLS) id 14.3.339.0; Wed, 27 Dec 2017 03:33:17 -0500 Received: from SPSCMX010.exchad.jpmchase.net ([169.254.3.142]) by SPSCDI005.exchad.jpmchase.net ([169.76.160.21]) with mapi id 14.03.0339.000; Wed, 27 Dec 2017 14:03:14 +0530 From: "Thacker, Dharam" To: John Blum , "user@geode.apache.org" Subject: Proposal on new features [Json Extension] Thread-Topic: Proposal on new features [Json Extension] Thread-Index: AdN+7VKHb7o6RgWVSEK8e5uowCF1wA== Date: Wed, 27 Dec 2017 08:33:13 +0000 Message-ID: <7D6D2BDAAF4BEC4EA1BE9AFB264505983949397B@SPSCMX010.exchad.jpmchase.net> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [10.119.148.133] x-jpmc-sendertrust: JPMC Content-Type: multipart/alternative; boundary="_000_7D6D2BDAAF4BEC4EA1BE9AFB264505983949397BSPSCMX010exchad_" MIME-Version: 1.0 X-DLP-FWD: Yes X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFprCJsWRmVeSWpSXmKPExsWy0tXaSvdclHOUwfObxhbnXqxlsZi++gGL A5PH4dn3GD2O397CHsAU1cBok5iXl1+SWJKqkJJanGyr5JJZnJyTmJmbWqSgq+BVkJtflJ6Y p5ecn6ukkJliq2SipFCQk5icmpuaV2KrlFhQkJqXomTHpYABbIDKMvMUUvOS81My89JtlTyD /XUtLEwtdQ2V7EIyMosVclOLixPTE/ayZzTd/shaMG8TY8XvRU4NjJPnMXYxcnJICJhIrFp9 ir2LkYtDSOAwo8T7v79ZIJwDjBKz+lpYIZxNjBJTvh5gAmlhEzCVaHm3lbmLkYNDRCBY4uhb eZCwsICRxKeTL1lAbBEBc4ltG5oYIWw9iSVdX8FaWQRUJT63P2UDsXkFQiTW/f8GVs8oICbx /dQasBpmAXGJW0/mM0FcJyCxZM95ZghbVOLl43+sELaSxJ0nU6HiEhI3Hs9mg+jNl9jyYy8L xHxBiZMzn4DZQgIaEvNmNzOCnCwhwCexva16AqPoLCTbZiHpnoWkGyKuJ3Fj6hQ2CFtbYtnC 18wQtq7EjH+HWJDFFzCyr2KULM5NzzDUKy4r0ist1ssqyE3OSCxO1ctLLdnECEo0Na1pOxjv LrU9xCjAwajEw5twzClKiDWxrLgy9xCjBAezkgiv8zSHKCHelMTKqtSi/Pii0pzU4kOMExmB ITiRWUo0OR+YBvNK4g2NzSwMTE0NDSwMjUxoKawkzms8NSpCSCAdmIazU1MLUotgjmLi4JRq YFx5WWe1jsLfp9lPVFeUPz3GknhS9I9tar7umjfWPw5MbEptVOP2UNSonSfj8bfaz8ZUP3lp 8aOClOUOOY07PydEXpFMUX/of/Kz+Mz5bSxXjJO1F6yKubfe4b4ZXwNvyhWB/6WbN5qdOZ/2 5fgR54Vfvmd/thE9aBcVJzb7X5Haka+3rlUvE1diKc5INNRiLipOBADwGC8vpwMAAA== archived-at: Wed, 27 Dec 2017 08:33:33 -0000 --_000_7D6D2BDAAF4BEC4EA1BE9AFB264505983949397BSPSCMX010exchad_ Content-Type: text/plain; charset="iso-8859-1" content-transfer-encoding: quoted-printable Hi John & Team, I would like to discuss about proposal for new feature with regards to JSON= documents in GEODE mainly intended for spring-data-geode. What is available and trade offs? I am aware of following ways but let me explain! 1. Region [JSONFormatter.fromJson(json)] Description: This is powerful api to store literally any JSON document into region withou= t worrying about underlying schema. But it comes up with several tradeoffs a= s explained below, a. As there is no entity defined pojo, one will not be able to take adv= antage of recent pdx-jdbc mappings possibly going to be part of Apache-Geod= e-1.4.0 b. As it is PdxInstance, implementing "Delta" interface is tricky and c= omplex to get information on what has changed inside document. c. Colocation strategy is not possible as explained in "point - a" due= to missing pojo/well defined schema. One has to create REPLICATED REGION if= he is intended to use "EQUI JOIN" with other regions d. :) 2. [Useful when you know the schema a= nd play around with your entity objects] Description: Possibly using AOP but at high level it encapsulates "JSONFormatter" api but= stricter part "It needs entity/schema/pojo" I also know about recent introduction of "@EnableEntityDefinedRegions" to av= oid cumbersome definition of all region beans in config. I think that's an annotation which can help if extended to overcome above ex= plained limitations at some extent. =F0 Now a days lot of databases (Oracle 12c ,MariaDB 10.2 etc.) provides na= tive support for json storage and powerful json functions to query underlyin= g data =F0 Even ORM frameworks works well and compatible enough as shown in below= example, @Entity class Customer { @Id private String cid; @Lob private String customerDetails; } =F0 As shown above, above class will be loaded with full json contents from= database but now there is complete disconnect with NO-SQL store to RDBMS sy= stem. =F0 There is no good way you can play around "customerDetails" json with ne= sted structure in Geode if you store your POJO as it is in "/Customer" regio= n =F0 The way we use as of now [customerTemplate.put(customer.getCid(),JsonFo= rmatter.fromJson(customer.getCustomerDetails())] Possible Extension =F0 As per my understanding, I see that possibly extension of "@EnableEntit= yDefinedRegions" could help here =F0 It would require few more hooks to properly auto convert @Lob in JPA vs= @Json in GEODE to auto convert it to PDXInstance when storing to region =F0 It would also require few more annotations to define co-location of sim= ilar regions so that one can use all possible region types Hope it makes sense! Let us know your thoughts as well. Thanks & Regards, Dharam This message is confidential and subject to terms at: http://www.jpmorgan.co= m/emaildisclaimer including on confidentiality, legal privilege, viruses and= monitoring of electronic messages. If you are not the intended recipient, p= lease delete this message and notify the sender immediately. Any unauthorize= d use is strictly prohibited. --_000_7D6D2BDAAF4BEC4EA1BE9AFB264505983949397BSPSCMX010exchad_ Content-Type: text/html; charset="iso-8859-1" content-transfer-encoding: quoted-printable

Hi John & Team,<= o:p>

 

I would like to disc= uss about proposal for new feature with regards to JSON documents in GEODE m= ainly intended for spring-data-geode.

 

What is ava= ilable and trade offs?

 

I am aware of follow= ing ways but let me explain!

 

1.&n= bsp;     Region <S= erializable,PDXInstance> [JSONFormatter.fromJson(json)]=

 

Description:

This is powerful api to store literally any JSON document into region without worrying about underlying schema. But it comes up with several tradeoffs as explained below,

 

a. &nb= sp;    As there is= no entity defined pojo, one will not be able to take advantage of recent pdx-jdbc  mappings possibly going to be part of Ap= ache-Geode-1.4.0

b. &nb= sp;    As it is Pdx= Instance, implementing “Delta” interface is tricky and complex t= o get information on what has changed inside document.

c. &nb= sp;     Colocation s= trategy is not possible as explained in “point – a” due to= missing pojo/well defined schema. One has to create REPLICATED REGION if he= is intended to use “EQUI JOIN” with other regions

d. &nb= sp;    <There= might be few more I am not aware of but you can help> J

 

2.&n= bsp;     <gfe-data= :json-region-autoproxy/> [Useful when you know the schema and play around= with your entity objects]

 

Description:

Possibly using AOP but at high level it encapsulates “JSONForma= tter” api but stricter part “It needs entity/schema/pojo”<= o:p>

 

 

I also know about re= cent introduction of “@EnableEntityDefinedRegions” to avo= id cumbersome definition of all region beans in config.

I think that’s= an annotation which can help if extended to overcome above explained limita= tions at some extent.

 

=F0  Now a days l= ot of databases (Oracle 12c ,MariaDB 10.2 etc.) provides native support for= json storage and powerful json functions to query underlying data

=F0  Even ORM fra= meworks works well and compatible enough as shown in below example,

 

@Entity<= /o:p>

class Customer {

          @Id
          private String cid;

          @Lob

          private String customerDetails;<= /span>

}

   &n= bsp;       

=F0  As shown abo= ve, above class will be loaded with full json contents from database but now= there is complete disconnect with NO-SQL store to RDBMS system.

=F0  There is = no good way you can play around “customerDetails” json with= nested structure in Geode if you store your POJO as it is in “/Custom= er” region

=F0  The way we u= se as of now [customerTemplate.put(customer.getCid(),JsonFormatter.fromJs= on(customer.getCustomerDetails())]

 

Possible Ex= tension

 

=F0  As per my un= derstanding, I see that possibly extension of “@EnableEntityDefinedRegion= s” could help here

=F0  It would req= uire few more hooks to properly auto convert @Lob in JPA vs @Json in GEODE t= o auto convert it to PDXInstance when storing to region

=F0  It would als= o require few more annotations to define co-location of similar regions so t= hat one can use all possible region types

 

Hope it makes sense!=

 

Let us know your tho= ughts as well.

 

Thanks & Regards= ,

Dharam

 

This message is confidential and subject to terms at: http://www.jpmorgan.com/emaildisclaimer= including on confidentiality, legal privilege, viruses and monitoring of el= ectronic messages. If you are not the intended recipient, please delete this= message and notify the sender immediately. Any unauthorized use is strictly= prohibited.

--_000_7D6D2BDAAF4BEC4EA1BE9AFB264505983949397BSPSCMX010exchad_--