From dev-return-27820-archive-asf-public=cust-asf.ponee.io@geode.apache.org Thu Jan 25 22:33:46 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id C7447180651 for ; Thu, 25 Jan 2018 22:33:46 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id B72A9160C3D; Thu, 25 Jan 2018 21:33:46 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id D61A8160C17 for ; Thu, 25 Jan 2018 22:33:45 +0100 (CET) Received: (qmail 20136 invoked by uid 500); 25 Jan 2018 21:33:44 -0000 Mailing-List: contact dev-help@geode.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@geode.apache.org Delivered-To: mailing list dev@geode.apache.org Received: (qmail 20123 invoked by uid 99); 25 Jan 2018 21:33:44 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 25 Jan 2018 21:33:44 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id B209D1A0D4A for ; Thu, 25 Jan 2018 21:33:43 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 1.98 X-Spam-Level: * X-Spam-Status: No, score=1.98 tagged_above=-999 required=6.31 tests=[DKIM_SIGNED=0.1, DKIM_VALID=-0.1, HTML_MESSAGE=2, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01] autolearn=disabled Authentication-Results: spamd2-us-west.apache.org (amavisd-new); dkim=pass (2048-bit key) header.d=pivotal-io.20150623.gappssmtp.com Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id rLDYg2Lf9yRa for ; Thu, 25 Jan 2018 21:33:40 +0000 (UTC) Received: from mail-lf0-f43.google.com (mail-lf0-f43.google.com [209.85.215.43]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTPS id 64D365F23E for ; Thu, 25 Jan 2018 21:33:39 +0000 (UTC) Received: by mail-lf0-f43.google.com with SMTP id x196so11664578lfd.12 for ; Thu, 25 Jan 2018 13:33:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=pivotal-io.20150623.gappssmtp.com; s=20150623; h=mime-version:in-reply-to:references:from:date:message-id:subject:to; bh=88ToKDjtAobOCQTUn+4lWbHes+2FlibtcZkLtDdlWAo=; b=XR1atFDPxwPmDv/njyImaBgZ6e73QLJNDdeQWNwMJjpWkzr6g5ueY13gB6aJI1YNfl G/mmBBr8mZeEiAAMGhl/eZQrT6lowrLAQvHEnYfOsGA8JMsvsM32Ox+3SURL6U0jneB2 0LrXlc/fG58JZkIkO6Br6SxgYw4bEPY4Q55rnpsPXtSRoFbJP3MNqPTwfzBf5hZe5xze R5IB0+tSKlQTggxZfwKg8EyCwQY22+wwElJPWhfxrjV8gZo3/0Bz4pdA9cKW4FcBMt7K 1EXrQp6jkS5y51idVLW9CnuGUR3NiE2YIylHwqTuILLHsmYdb/Tdlug+fH4bODTpRgAC IJeg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to; bh=88ToKDjtAobOCQTUn+4lWbHes+2FlibtcZkLtDdlWAo=; b=PrDpC8e1qoOchXejxCP/yBMVjQE+uHBOca24doIFGSg/49tE23yHEy47Mx0uIaCBY0 mAFH8sWHS3Ai3jA7ek2UHMz3EK4Ohx9qvahVNEoEBJwD2HNYDLaURwgL+0GveqY7dL1s geqTm6Skd48DAy2vfFmnTOKIqpqpW55tcTdu0erCq0Bea32NGMRuMjRzfP1/vF3h5AWP 584BkZt8Sh1M/1NOGwHVLFkWimsCpUa2OIzfV2xn7Lr1687xEu+t2h/5FHJTBwEBZOLm XIALBXq/Y3vJtshLtEPwYrq1/2gccrUGfy3SJvGXOw+ELX7VwWQT+11uxIBuuMLVvcM4 jTdQ== X-Gm-Message-State: AKwxytc9EKHOmqAVMI8LctPAMzt4AhWqBbbj8IJ8Lg1aE9UzFlxnKRE9 LPgSYWT4Xd8/z49o3OZZ+4QoSxE4OWbN1u1oH/lQMALo X-Google-Smtp-Source: AH8x225EIdzZo9G5cR1EeOzf3iPuc30+Aq4YBPG63yXh5OAy1JbPfBrGH/erWQXSYoch4mkIcuHl8v/avdWh8R8x5Rw= X-Received: by 10.25.31.200 with SMTP id f191mr6745083lff.68.1516916011204; Thu, 25 Jan 2018 13:33:31 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.100.81 with HTTP; Thu, 25 Jan 2018 13:33:30 -0800 (PST) In-Reply-To: References: <2b69a46f-e852-af18-4fbb-5d10dfd0abd5@apache.org> <8870A9ED-410E-472E-92D9-4954B0589341@pivotal.io> From: John Blum Date: Thu, 25 Jan 2018 13:33:30 -0800 Message-ID: Subject: Re: Proposal: GEODE-4367 - Return PDXInstance when Domain Object can't be found To: geode Content-Type: multipart/alternative; boundary="001a11402c5400a3f40563a088df" --001a11402c5400a3f40563a088df Content-Type: text/plain; charset="UTF-8" *One more thing...* Regarding arguments passed to Functions, which can equally apply to return types as well, particularly in SDG, since Functions can be methods on a POJO, having strongly typed return values, for example ... @GemfireFunction(..) Customer someFunction(CustomerIdentifier id, PdxInstance someObject, Object someValue, ...) { .. } I had to handle this very problem, that is, when PDX is enabled and the Function either takes a PdxInstance or perhaps a Java type (e.g. Customer) and PDX bytes are sent; what then; how to handle? Another scenario that needs special consideration is, if the Function takes a generic argument, like Object, do you hand the user a PdxInstance (if PDX is in play), or does the framework try to convert the type (i.e. PdxInstance.getObject()) if possible, meaning the type is possibly on the CLASSPATH if the Function argument is strongly typed? Anyway, awhile back, I added some additional logic in SDG to more intelligently handle such scenarios [1]. While it isn't perfect, it is far more robust than what existed previously (thus, "building up"). The point is, the user did not need to do anything (i.e. extra configuration) in order for SDG to do the right thing for the user, covering 80% of the most common scenarios. -j [1] https://github.com/spring-projects/spring-data-geode/blob/2.0.3.RELEASE/src/main/java/org/springframework/data/gemfire/function/PdxFunctionArgumentResolver.java On Thu, Jan 25, 2018 at 1:16 PM, John Blum wrote: > To Swap's point about making ReflectionBasedAutoSerializer the default > (which I think is a good idea), comparably, in SDG, I have made SDG's > o.s.d.g.mapping.MappingPdxSerializer [1] the default when PDX is enabled > (i.e. @EnablePdx). No need to explicitly declare/register it. SDG's > MappingPdxSerializer leverage's *Spring Data's* powerful mapping > infrastructure to convert Java objects to PDX and back. This is even more > robust than Apache Geode's own ReflectionBasedAutoSerializer, but clearly > *Spring* specific. > > More details available here [2] (which as it turns out, needs to be > updated based on the latest developments, namely DATAGEODE-72 [3]). > > -John > > > [1] https://docs.spring.io/spring-data/geode/docs/current/api/org/ > springframework/data/gemfire/mapping/MappingPdxSerializer.html > [2] https://docs.spring.io/spring-data/geode/docs/current/reference/html/# > bootstrap-annotation-config-pdx > [3] https://jira.spring.io/browse/DATAGEODE-72 > > > On Thu, Jan 25, 2018 at 12:44 PM, Jacob Barrett > wrote: > >> >> >> > On Jan 25, 2018, at 12:06 PM, Anilkumar Gingade >> wrote: >> > >> > Jake, >> > >> > Interesting...Its the argument passed to function that is in >> deserialized >> > form, right? >> >> Yes, if pdx read serialized is false (default) the argument is >> deserialized to domain object or fails if no domain object found. If read >> serialized is true then all arguments that are PDX serialized and passed in >> as PdxInstance. Sett my this flag breaks all previous deployed functions >> that expected domain objects in the argument. >> >> -Jake >> >> >> > >> > -Anil. >> > >> > >> > >> > >> > >> > >> >> On Thu, Jan 25, 2018 at 10:12 AM, Jacob Barrett >> wrote: >> >> >> >> Anil, this wouldn't fix a function execution where a parameter sent to >> the >> >> function is a PDX serialized object. The function would be invoked on >> the >> >> server with the current global setting and file to deserialize the PDX >> to >> >> domain object. This is why the ask is for it to fall back to >> PdxInstance if >> >> the domain object is not deployed. >> >> >> >> -Jake >> >> >> >> On Thu, Jan 25, 2018 at 9:56 AM Anilkumar Gingade > > >> >> wrote: >> >> >> >>> Internally, there is an option to override read-serialized flag (to >> >> true); >> >>> the query engine and other components uses this to keep the data in >> >>> serialized form and work with PdxInstance... >> >>> >> >>> public static void setPdxReadSerialized(Cache cache, boolean >> >>> readSerialized); >> >>> >> >>> We had discussed, making this as a public api...so that any thread >> that >> >> can >> >>> work on PdxInstance can take advantage of it... >> >>> >> >>> -Anil. >> >>> >> >>> >> >>> On Thu, Jan 25, 2018 at 9:42 AM, Jacob Barrett >> >>> wrote: >> >>> >> >>>> Bruce, the flag only applies to values serialized with PDX, >> >>>> DataSerializable objects are not effected by this property. >> >>>> >> >>>> I think there is some real value here as a stop gap until we have a >> >>> better >> >>>> solution in Geode 2 where the user can have a per request context >> that >> >>>> specifies what return type they would like. Consider the user that >> has >> >> an >> >>>> existing application that uses domain objects but wants to deploy >> >> another >> >>>> application that doesn't to the same Geode cluster. The only option >> is >> >> to >> >>>> either have all PDX deserialize to domain objects or all returned as >> >>>> PdxInstance. One of the two applications will not work without >> >>>> modification. Changing the behavior described by Addison splits the >> >>>> difference. If the application is, like it is by default, configure >> to >> >>>> deserialize PDX to the domain object but the domain object is not >> >>> deployed >> >>>> it will now give back the PDX instance rather than failing. >> >>>> >> >>>> An explicit use case is user that has both a Java and .NET >> application. >> >>> The >> >>>> .NET application does not have any Java domain objects to deploy to >> the >> >>>> server but does want to query or run server side functions. The Java >> >>>> application has deployed the domain objects to the server and >> >> distributed >> >>>> functions are written expecting those domain objects on the server. >> The >> >>>> user would have to create Java domain objects for the .NET >> application >> >> or >> >>>> modify their Java application to expect PdxInstance. >> >>>> >> >>>> >> >>>> -Jake >> >>>> >> >>>> >> >>>> On Thu, Jan 25, 2018 at 7:38 AM Bruce Schuchardt < >> >> bschuchardt@apache.org >> >>>> >> >>>> wrote: >> >>>> >> >>>>> +1 >> >>>>> >> >>>>> I've found the current read-serialized property to be pretty >> useless. >> >>>>> >> >>>>> Having said that, what if the value isn't actually in serialized >> form >> >>> in >> >>>>> the local cache? Is Geode supposed to serialize it & return it? >> >> What >> >>>>> if it isn't PDX-serialized? Do we return a byte array? >> >>>>> >> >>>>> >> >>>>>> On 1/24/18 12:21 PM, Dan Smith wrote: >> >>>>>> Is this really just a workaround for the fact that the >> >>> read-serialized >> >>>>> flag >> >>>>>> applies to the whole cache? I can see that if you have mix of >> >> regions >> >>>>> with >> >>>>>> and without domain classes on the server you might want this >> >> feature. >> >>>> Can >> >>>>>> you provide some more background on your use case? >> >>>>>> >> >>>>>> IMO we should get rid of read-serialized in favor of APIs that let >> >>> the >> >>>>> user >> >>>>>> decide whether they get a domain class or a PdxInstance. >> >>>>>> >> >>>>>> -Dan >> >>>>>> >> >>>>>> On Wed, Jan 24, 2018 at 9:58 AM, Galen O'Sullivan < >> >>>> gosullivan@pivotal.io >> >>>>>> >> >>>>>> wrote: >> >>>>>> >> >>>>>>> Hi Addison, >> >>>>>>> >> >>>>>>> What kind of setup do you have that is causing you to have PDX >> >>>>> serialized >> >>>>>>> objects that cannot be deserialized? Do you have classes that are >> >>>>> present >> >>>>>>> on some servers but not others? >> >>>>>>> >> >>>>>>> This change would break the contract of region.get() , which is >> >> that >> >>>> it >> >>>>>>> returns a value of a type that can be put into the region. >> >>>>>>> >> >>>>>>> Returning something that isn't what the user is expecting to be in >> >>> the >> >>>>>>> region would require users to check for PdxInstances every time >> >> they >> >>>>> get a >> >>>>>>> value -- even though the type annotations say that you can't get a >> >>>>>>> PdxInstance back (except for Region ). >> >>>>>>> >> >>>>>>> I think it would be better to have a second API that allows users >> >> to >> >>>> get >> >>>>>>> and put PdxInstance objects in regions. That way, if they want to >> >>>> handle >> >>>>>>> the exceptional case where they have a serialized object that >> >> can't >> >>> be >> >>>>>>> deserialized, they can catch the ClassNotFound exception and get >> >> the >> >>>>>>> underlying PdxInstance. >> >>>>>>> >> >>>>>>> I do think that the possibility of a ClassNotFoundException should >> >>> be >> >>>>>>> documented in the Region API. >> >>>>>>> >> >>>>>>> Cheers, >> >>>>>>> Galen >> >>>>>>> >> >>>>>>> On Tue, Jan 23, 2018 at 2:56 PM, Addison Huddy > >>> >> >>>>> wrote: >> >>>>>>> >> >>>>>>>> Hi Geode Devs, >> >>>>>>>> >> >>>>>>>> I'm proposing the following change to how we handle >> >> deserialization >> >>>>> when >> >>>>>>>> Domain Objects can't be found and pdx-serialize=false. >> >>>>>>>> >> >>>>>>>> https://issues.apache.org/jira/browse/GEODE-4367 >> >>>>>>>> >> >>>>>>>> Looking forward to the discussion. >> >>>>>>>> >> >>>>>>>> \ah >> >>>>>>>> >> >>>>> >> >>>>> >> >>>> >> >>> >> >> >> > > > > -- > -John > john.blum10101 (skype) > -- -John john.blum10101 (skype) --001a11402c5400a3f40563a088df--