ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Kuznetsov <akuznet...@gridgain.com>
Subject Re: Cassandra cache store [IGNITE-1371]
Date Fri, 29 Jan 2016 06:12:31 GMT
I like Val suggestion.

Also in this case we could move external things like Kryo into separate
module/dependency (for those who do not want to depend on Kryo).
This could be "ignite-cassandra-kryo" module that will contain smth. like
"KryoPersistenceCallback".

Thoughts?


On Fri, Jan 29, 2016 at 10:27 AM, Valentin Kulichenko <
valentin.kulichenko@gmail.com> wrote:

> Alexey, Igor,
>
> Binary format is internal format used by Ignite, it can't be used directly
> or outside of Ignite. So there is no way and no reason to have special
> binary serializer in Cassandra store implementation. If user wants to save
> BinaryObject as a BLOB to the store (to later load it back to Ignite), he
> can set CacheConfiguration.setStoreKeepBinary(true) flag and cache will
> give BinaryObject to the store without deserializing it.
>
> Actually, I'm looking at the current API and I don't like
> PersistenceStrategy enum and Serializer interface. I understand what you
> try to achieve here and it makes sense to me, but the abstraction is not
> generic enough in my opinion. We give user only two options, but what if he
> needs something else? For example, he may need to add special processing
> for some specific fields (make some conversions, etc.).
>
> I propose to change it a bit and do something like this:
>
> interface PersistenceCallback {
>     CassandraData onWrite(K key, V value);
>     IgniteBiTuple<K, V> onRead(CassandraData);
> }
>
> CassandraData is an entity that Cassandra should be able to write and load
> (essentially, set of column values). We can reuse something from Cassandra
> API here if that's possible (I'm not familiar enough with Cassandra to make
> specific suggestions right away) or create our own.
>
> We will have a default implementation that will introspect POJOs. The one
> that converts value to BLOB can be also provided out of the box. And user
> is free to implement his own logic.
>
> All namings and API itself are arguable, of course, but I hope that idea is
> clear.
>
> Thoughts?
>
> On Thu, Jan 28, 2016 at 12:41 AM, Alexey Kuznetsov <
> akuznetsov@gridgain.com>
> wrote:
>
> > During review I faced the problem, that I have not enough experience to
> > answer.
> >
> > So, it will be great, if someone who has deep experience with Binary
> > marshaller could help us.
> >
> > ---------------------------
> >
> > > How about such limitations of Binary Marshaller:
> >
> > > 1) Fields or types with the same name hash are not allowed.
> > > 2) BinaryObject format does not allow same field names on different
> > levels of a class hierarchy.
> > > 3) Externalizable interface is ignored by default. If BinaryObject
> format
> > is used, Externalizable
> > >  classes will be written the same way as if they were Serializable,
> > without writeExternal() and
> > > readExternal() methods. If for some reason this does not work for you,
> > you should implement
> > > Binarylizable interface for your classes, plug in a custom
> > BinarySerializer or switch to the
> > > OptimizedMarshaller.
> >
> > > There are no such limitations in Kryo and Java serialization.
> >
> > > The next concern is that you need *Ignite Core* module, which is rather
> > huge (about 7.3MB) if
> > >  you want to build ETL script which will consume data persisted into
> > Cassandra by Ignite Binary
> > >  Marshaller.
> >
> > > By the way, does Ignite plan to support backward compatibility for data
> > persisted using Binary
> > >  Marshaller? I mean the situation when some objects were persisted into
> > Cassandra using old
> > >  version of Binary Marshaller and then Ignite cluster was upgraded to
> new
> > version. Kryo and
> > >  Java serialization handles this situation and provides backward
> > compatibility.
> >
> > > May be it's better just to add one more serializer implementation which
> > will use Binary Marshaller?
> >
> > > By the way are there any samples in the code how to use Binary
> Marshaller
> > just to
> > > serialize/deserialize arbitrary object? Binary Marshaller documentation
> > says that all such
> > >  operation performed internally inside Ignite when we using
> BinaryObject,
> > but in my case I am
> > >  interested in rather low-level serialization/deserialization API/
> >
> > Regards,
> > Igor Rudyak
> >
> > --------------------------------------------
> > Hi Igor!
> >
> > I moved discussion to dev list.
> >
> > >>> The next concern is that you need *Ignite Core* module
> > We could not avoid adding this module because All base cache store
> classes
> > declared in that module, so I think this module will *be always
> imported*.
> >
> > As for other points, I hope community will help us.
> >
> > --
> > Alexey Kuznetsov
> > GridGain Systems
> > www.gridgain.com
> >
>



-- 
Alexey Kuznetsov
GridGain Systems
www.gridgain.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message