ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Romain Gilles <romain.gil...@gmail.com>
Subject Re: OSGi migration may require a special marshaller
Date Tue, 03 Nov 2015 10:38:06 GMT
Hi Raul,
 - Do you plan to use the TCCL when marshalling in OSGi env? If yes how?
 - I like the point 2. Maybe for a graph of object that come from different
packages / bundles you may have to recursively capture the package version
of the individual element of your graph.
 - For the point 3 I wonder if it will not over-complexify the solution. As
a developer you will have to think about it. And it is not obvious in the
code where things are serialized or not. You may use lambda in your
application code therefore the current package become what you call a DTO
package. The current state of ignite does not enforce you to specify it for
"classical" classloading application. If you introduce this extra step for
OSGi ready application you will maybe discourage people to use OSGi.

To comeback to our solution. We start we a strong assumption: our cluster
is homogeneous in term of code so, of course, it simplify our life :).

BTW if you can introduce an extension point in the class resolution
mechanism it can be interesting for end user in order to allow them to
optimize it based on there specific use cases.

Romain.

Le mar. 3 nov. 2015 à 00:21, Raul Kripalani <raul@evosent.com> a écrit :

> Hi Andrey,
>
> Thanks for the participation in this topic.
>
> I don't like the solution to incorporate the bundle symbolic name in the
> serialised form. Nothing guarantees that the classdef will be located under
> the same bundle in both source and target machines. We also have to take
> into account that OSGi allows for multiple versions of the same
> bundle/packages to co-exist in the same  container. So it becomes more
> complex.
>
> Using the TCCL should work when serialising, but it'll probably be of no
> use when deserialising on the other end.
>
> I need to enhance Ignite to:
>
> 1. Use the TCCL when marshalling on one end.
> 2. Incorporate the package version of the class in the serialised form if
> Ignite is running in an OSGi environment.
> 3. On the receiver end, discover cache entities / DTOs in all bundles
> through a custom OSGi manifest header or the like, as I explained before.
> Package version must be taken into account.
>
> What do you think?
>
> Raúl.
> On 2 Nov 2015 17:25, "Andrey Kornev" <andrewkornev@hotmail.com> wrote:
>
> > Raul,
> >
> > The fundamental hurdle we need to jump over to make Ignite OSGi-enabled
> is
> > the marshalling. More specifically the issue is with deserialization of
> the
> > classes that are provided by the bundles other than the Ignite bundle
> > itself.
> >
> > When the Ignite transport layer receives a message it needs to figure out
> > how to deserialize the bytes and for that it needs to know the bundle
> that
> > provides the class to be deserailized. At this point TCCL is of no use.
> To
> > make things more complex, the class may contain other classes that come
> > from other bundles, and so on recursively. This means that each object in
> > the hierarchy must be serialized with its bundle name (or bundle id), so
> > that the deserializer will then be able to correctly resolve the class
> > while traversing the object hierarchy during deserialization.
> >
> > Unfortunately, Ignite's OptimizedMarshaller is lacking the ability to
> plug
> > in a custom class resolver. Romain's solution was to use Kryo that does
> > provide a way to customize class resolution. It has solved the problem of
> > capturing the bundle info and he was able to successfully run Ignite as a
> > bundle in an OSGi container (after some repackaging and inclusion of the
> > manifest). But Kryo-based marshalling introduced a lot of complexity to
> the
> > code and incorrect use of Kryo's numerous serializers caused some weird
> > hard-to-debug issues in the Ignite core (like duplicate cache entries due
> > to incorrect marshalling of the GridDhtPArtitonFullMap class -- go
> > figure!). Overall the Kryo-based solution is brittle and hard to
> maintain.
> >
> > I feel the correct solution to OSGi problem would be to
> > 1) enhance the OptimizedMarshaller to allow custom class resolution.
> > 2) provide an OSGi-enabled OptimizedMarshaller (in addition to the
> > original one) to be used in OSGi environment.
> >
> > Regards
> > Andrey
> >
> > > From: raulk@apache.org
> > > Date: Mon, 2 Nov 2015 12:41:47 +0000
> > > Subject: Re: OSGi migration may require a special marshaller
> > > To: dev@ignite.apache.org
> > >
> > > Hi Romain,
> > >
> > > I'm working on the OSGi compatibility of Ignite. I appreciate your
> input.
> > >
> > > I'm thinking about the situation you describe and I wonder if you're
> > > exporting Ignite as an OSGi service which is then consumed from other
> > > bundles. Under this situation, it would be quite easy to reproduce the
> > > behaviour you describe if Ignite is not resolving classes via the TCCL.
> > > Need to dig deeper into that.
> > >
> > > Off the top of my head, there are two alternatives to solve it:
> > >
> > > 1. Use the TCCL for marshalling/unmarshalling (if not already used) –
> we
> > > gotta be wary of possible regressions.
> > > 2. Create a special OSGi header Ignite-Export-Package so that bundles
> > > containing DTOs can expose packages to Ignite's marshallers.
> > >
> > > Regards,
> > >
> > > *Raúl Kripalani*
> > > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data
> and
> > > Messaging Engineer
> > > http://about.me/raulkripalani |
> http://www.linkedin.com/in/raulkripalani
> > > http://blog.raulkr.net | twitter: @raulvk
> > >
> > > On Mon, Nov 2, 2015 at 9:56 AM, Gilles, Romain <
> romain.gilles@misys.com>
> > > wrote:
> > >
> > > > Hi all,
> > > >
> > > > I'm really interested in this issue:
> > > > https://issues.apache.org/jira/browse/IGNITE-1270 . We some stuff to
> > make
> > > > it work in our osgi environment. The main issue for us now is the
> > > > serialization. I think it you will have to rework the
> > OptimizedMarshaller
> > > > or any other marshaller that works with object that come from outside
> > your
> > > > class space.
> > > >
> > > > We have try kryo that works. Kryo provide an extension point in order
> > to
> > > > resolve the classes:
> > > >
> >
> https://github.com/EsotericSoftware/kryo/blob/master/src/com/esotericsoftware/kryo/ClassResolver.java
> > > > . With this extension we are able to solve the problem of external
> > classes.
> > > > The only issue with kryo is that some classes need a certain care in
> > the
> > > > serialization process and therefore a specialized serializer.
> > > >
> > > > So I would like to know from the community what do think of changing
> > the
> > > > way the optimized marshaller works or introducing the support of yet
> > > > another marshaller based on a kryo like technology?
> > > >
> > > >
> > > > Thanks in advance,
> > > >
> > > >
> > > > Best regards,
> > > >
> > > >
> > > > Romain.
> > > >
> > > >
> > > > PS: I'm ready to help in the both cases.
> > > > "Misys" is the trade name of the Misys group of companies. This email
> > and
> > > > any attachments have been scanned for known viruses using multiple
> > > > scanners. This email message is intended for the named recipient
> only.
> > It
> > > > may be privileged and/or confidential. If you are not the named
> > recipient
> > > > of this email please notify us immediately and do not copy it or use
> > it for
> > > > any purpose, nor disclose its contents to any other person. This
> email
> > does
> > > > not constitute the commencement of legal relations between you and
> > Misys.
> > > > Please refer to the executed contract between you and the relevant
> > member
> > > > of the Misys group for the identity of the contracting party with
> > which you
> > > > are dealing.
> > > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message