Mailing-List: contact dev-help@cayenne.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cayenne.apache.org
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
Subject: Re: Cayenne ROP Improvements
From: Andrus Adamchik <andrus@objectstyle.org>
In-Reply-To: <CAAWAmOfg30npv7KPjRy9MPzX-XWxVK2m9c_fd0FWx9P1jE0T8g@mail.gmail.com>
Date: Thu, 16 Jun 2016 16:23:45 +0300
Content-Transfer-Encoding: quoted-printable
Message-Id: <2AE81886-40A1-4B7B-81E2-EAD4FEAAC9F0@objectstyle.org>
References: <CAAWAmOePgD_x9xp3s1iY+3qWAwTgsqdW8kpomBjwNY0_5tvJEQ@mail.gmail.com> <33ee68ff-b75e-b1e2-7be0-d83264755b77@maniatis.org> <CAAWAmOc5JRhvpxGbPmzq5wbJj-0rBngyirHeRb=PNbCrw7Oi0w@mail.gmail.com> <CAAWAmOecFpP98SUg8bk50b9=nqTX5UPsvw=-Nw5+MRoO_YwVUg@mail.gmail.com> <7C2A950F-BC8A-4B83-97E3-B2BDDAD9B7AA@objectstyle.org> <34307cec-a5b6-1f0e-66a8-60a5aec3cca4@maniatis.org> <F3BC25B7-198F-4741-A4ED-A1FA5C77D166@objectstyle.org> <f428cf5e-437f-13ec-7940-341f9319eee1@maniatis.org> <09EF91F4-FA51-4F31-9C78-7D34ED1FDD37@objectstyle.org> <CAAWAmOfg30npv7KPjRy9MPzX-XWxVK2m9c_fd0FWx9P1jE0T8g@mail.gmail.com>
To: dev@cayenne.apache.org
archived-at: Thu, 16 Jun 2016 13:23:55 -0000

> Please, let me know what do you think about all this things?

Sounds reasonable to me.=20

> Finally, as Cayenne is becoming more modular, it will be helpful to =
have
> server and client test jars. It is sometimes frustrating to write
> integration tests for new modules.

No objections here. Let's make sure we set deploy plugin "skip" property =
to "true". We don't want them to be part of releases.

Andrus


> On Jun 13, 2016, at 11:40 AM, Savva Kolbachev <s.kolbachev@gmail.com> =
wrote:
>=20
> After some time I think that my current HTTP/2 ROP implementation for
> Cayenne [1] is cumbersome and complicated. I want to refactor it and =
make
> it as clear as possible, so it will be simple to use out of box.
> So, I suggest to rename module to something like =
"cayenne-jetty-client" and
> implement only two ROPConnectors: JettyHttpConnector and
> JettyHttp2Connector for HTTP/1.1 and HTTP/2 accordingly. Both will use
> high-level Jetty Client API.
> If someone wants something specific, they could provide their own =
similar
> implementation. I think the most common case for this is to share one
> HttpClient between ROP and other parts of a project.
> In addition, we will be open to make more modules for other clients, =
like
> Apache HTTP Client, Netty, OkHttp and etc.
>=20
> BTW, I got rid of Hessian ServiceContext in ROPServlet [2] by =
implementing
> something very similar in Cayenne
> =
https://github.com/apache/cayenne/commit/85852f92ee05f5e1ae70dae7315beda32=
f2b4077
> So now Hessian is used only for serialization purposes. As we want to =
move
> all ROP functionality to the separate module and Hessian serialization =
has
> problems with Java8, we should decide which serialization service =
should be
> used by default. For example Protostuff or Java serialization. Hessian =
one
> we could remain as pluggable.
>=20
> Finally, as Cayenne is becoming more modular, it will be helpful to =
have
> server and client test jars. It is sometimes frustrating to write
> integration tests for new modules.
>=20
> Please, let me know what do you think about all this things?
>=20
> [1] https://github.com/apache/cayenne/pull/96
> [2] https://issues.apache.org/jira/browse/CAY-2090
>=20
>=20
> 2016-05-06 3:29 GMT+03:00 Andrus Adamchik <andrus@objectstyle.org>:
>=20
>> Good idea. The server part should probably be split from =
cayenne-server.
>>=20
>> Andrus
>>=20
>>> On May 5, 2016, at 8:26 PM, Aristedes Maniatis <ari@maniatis.org> =
wrote:
>>>=20
>>> Perhaps the whole of ROP becomes an optional module?
>>>=20
>>> Ari
>>>=20
>>>=20
>>> On 6/05/2016 10:18am, Andrus Adamchik wrote:
>>>> It's been a while since I touched the ROP code. Back in the day =
Java
>> serialization "kind of worked", but not completely. So you are =
probably
>> right that it is not a real option. I am just trying to avoid new
>> dependencies (even optional) on third-party libs in the Cayenne core. =
So
>> perhaps we can simply leave out any "default" serialization and =
always
>> require an explicit serialization provider.
>>>>=20
>>>> Andrus
>>>>=20
>>>>> On May 5, 2016, at 8:12 PM, Aristedes Maniatis <ari@maniatis.org>
>> wrote:
>>>>>=20
>>>>> Maybe I'm not understanding correctly, but I don't think Java
>> serialisation has been implemented in ROP. The work Dima did was to =
move
>> away from the Hessian servlet stuff for making the HTTP connection, =
to
>> plain Java with the option for plugging in Jetty libraries for =
HTTP/2.
>>>>>=20
>>>>> The work Savva did just now was to use protostuff for =
serialisation,
>> but I'm not sure what's now needed if we wanted plain Java =
serialisation or
>> whether that's even possible without some sort of library to handle =
an
>> object graph with cycles.
>>>>>=20
>>>>> Or at least that's my understanding.
>>>>>=20
>>>>> Ari
>>>>>=20
>>>>>=20
>>>>> On 6/05/2016 9:55am, Andrus Adamchik wrote:
>>>>>> Thanks for clarification. I would say use Java serialization as a
>> default, and make it easy to plugin Hessian and Protostuff as =
separate
>> modules.
>>>>>>=20
>>>>>> A.
>>>>>>=20
>>>>>>> On May 5, 2016, at 5:39 PM, Savva Kolbachev =
<s.kolbachev@gmail.com>
>> wrote:
>>>>>>>=20
>>>>>>> Hi Andrus,
>>>>>>>=20
>>>>>>>> So which one is the default, Hessian or Java?
>>>>>>> We still use Hessian for serialization by default
>>>>>>>=20
>> =
https://github.com/apache/cayenne/blob/master/cayenne-server/src/main/java=
/org/apache/cayenne/rop/HessianROPSerializationService.java
>>>>>>> But we use java.net.URLConnection for establish connection and
>> sending
>>>>>>> messages from client to server
>>>>>>>=20
>> =
https://github.com/apache/cayenne/blob/master/cayenne-client/src/main/java=
/org/apache/cayenne/rop/http/HttpROPConnector.java
>>>>>>> So we have escaped from Hessian only in connectivity layer.
>>>>>>>=20
>>>>>>>> I don't have a problem with Protostuff being a recommended =
default,
>> but
>>>>>>> for dependency management purposes I'd rather we split all
>> third-party
>>>>>>> integrations in separate modules, and use whatever provider is
>> hooked up in
>>>>>>> runtime. Kind of what we do with Joda/Java8 extensions.
>>>>>>> I already did it in this way. I created separate module for
>> Protostuff
>>>>>>> serialization.
>>>>>>>=20
>>>>>>> As Hessian serialization has some troubles with Java8 types and
>> provide
>>>>>>> less efficient serialization than Protostuff, I suggest to use
>> Protostuff
>>>>>>> as default serialization service or to use Java serialization. =
So I
>> just
>>>>>>> suggest to escape from Hessian :)
>>>>>>>=20
>>>>>>> 2016-05-05 19:41 GMT+03:00 Savva Kolbachev =
<s.kolbachev@gmail.com>:
>>>>>>>=20
>>>>>>>> Hi Ari,
>>>>>>>>=20
>>>>>>>> Looks like Protostuff works faster than Protobuf in some cases. =
For
>>>>>>>> example Serializers (no shared refs) and Cross Lang Binary
>> Serializers
>>>>>>>> sections here
>> http://hperadin.github.io/jvm-serializers-report/report.html
>>>>>>>>=20
>>>>>>>> In our case we need to serialize graph of objects (Full Object =
Graph
>>>>>>>> Serializers section in link above). Protobuf can't do it out of =
the
>> box
>>>>>>>> but Protostuff can. In my implementation I use
>> protostuff-graph-runtime
>>>>>>>> which generates a schema from objects at runtime and caches it.
>>>>>>>>=20
>>>>>>>> Protostuff schema is something like .proto files but in Java:
>>>>>>>> http://www.protostuff.io/documentation/schema/
>>>>>>>> Runtime schema:
>> http://www.protostuff.io/documentation/runtime-schema/
>>>>>>>>=20
>>>>>>>> As you could see in benchmarks there is a small difference in
>> efficiency
>>>>>>>> between protostuff-graph and protostuff-graph-runtime. The =
ser/deser
>>>>>>>> overhead is related to runtime schema generation. The size =
penalty
>> is that
>>>>>>>> Protostuff adds class name for objects and than uses those for =
find
>>>>>>>> appropriate classes via reflection.
>>>>>>>> Hessian also adds fields names so the size of Hessian =
serialization
>> is
>>>>>>>> much bigger. In my small example with selection of 6 objects =
Hessian
>>>>>>>> serialization size is more than 2400 bytes while Protostuff =
runtime
>> is
>>>>>>>> about 800 bytes.
>>>>>>>>=20
>>>>>>>> If we don't want to have ser/deser and size overhead we could =
find
>> a way
>>>>>>>> to generate schemas via Velocity. And we should provide schemas =
for
>> some
>>>>>>>> Cayenne classes. But it will require a lot of efforts.
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> 2016-05-05 13:44 GMT+03:00 Aristedes Maniatis =
<ari@maniatis.org>:
>>>>>>>>=20
>>>>>>>>> On 5/05/2016 7:35pm, Savva Kolbachev wrote:
>>>>>>>>>> Protostuff (licensed under Apache 2.0 licence) is based on
>> Google's
>>>>>>>>>> Protocol-Buffers (Protobuf) but has some optimizations and =
some
>> cool
>>>>>>>>> things
>>>>>>>>>> like runtime serialization graph of objects (like Hessian). =
It
>> also
>>>>>>>>> could
>>>>>>>>>> generate schema on runtime so we shouldn't define .proto =
files
>> although
>>>>>>>>> it
>>>>>>>>>> might increase efficiency. It works faster than Hessian and =
could
>> handle
>>>>>>>>>> Java8 Date and Time types. Here is some benchmarks. Take a =
look
>> at Full
>>>>>>>>>> Object Graph Serializers section.
>>>>>>>>>> http://hperadin.github.io/jvm-serializers-report/report.html
>>>>>>>>>> https://github.com/eishay/jvm-serializers/wiki
>>>>>>>>>=20
>>>>>>>>> According to those benchmarks there appears to be no =
performance
>> or size
>>>>>>>>> penalty to using protostuff over protobuffers. Am I reading =
that
>> right?
>>>>>>>>>=20
>>>>>>>>> I don't really understand... doesn't the serialiser have to
>> construct a
>>>>>>>>> .proto definition and then include it in the message? So =
shouldn't
>> it be
>>>>>>>>> faster/smaller to predefine these?
>>>>>>>>>=20
>>>>>>>>> If we did, we could create them with velocity in the same way =
we
>> create
>>>>>>>>> Java _superclasses today. Fairly trivial I'm guessing.
>>>>>>>>>=20
>>>>>>>>> Ari
>>>>>>>>>=20
>>>>>>>>>=20
>>>>>>>>> --
>>>>>>>>> -------------------------->
>>>>>>>>> Aristedes Maniatis
>>>>>>>>> GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 =
102A
>>>>>>>>>=20
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>>=20
>>>>>>>> --
>>>>>>>> Thanks and Regards
>>>>>>>> Savva Kolbachev
>>>>>>>>=20
>>>>>>>=20
>>>>>>>=20
>>>>>>>=20
>>>>>>> --
>>>>>>> Thanks and Regards
>>>>>>> Savva Kolbachev
>>>>>>=20
>>>>>=20
>>>>> --
>>>>> -------------------------->
>>>>> Aristedes Maniatis
>>>>> GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A
>>>>=20
>>>=20
>>> --
>>> -------------------------->
>>> Aristedes Maniatis
>>> GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A
>>=20
>>=20
>=20
>=20
> --=20
> Thanks and Regards
> Savva Kolbachev