ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dmitriy Setrakyan <dsetrak...@apache.org>
Subject Re: Data compression in Ignite 2.0
Date Mon, 15 May 2017 16:02:25 GMT
Vyacheslav,

I think it is a bit premature to provide a PR without getting a community
consensus on the dev list. Please allow some time for the community to
respond.

D.

On Mon, May 15, 2017 at 6:36 AM, Vyacheslav Daradur <daradurvs@gmail.com>
wrote:

> I created the ticket: https://issues.apache.org/jira/browse/IGNITE-5226
>
> I'll prepare a PR with described solution in couple of days.
>
> 2017-05-15 15:05 GMT+03:00 Vyacheslav Daradur <daradurvs@gmail.com>:
>
> > Hi, Igniters!
> >
> > Apache 2.0 is released.
> >
> > Let's continue the discussion about a compression design.
> >
> > At the moment, I found only one solution which is compatible with
> querying
> > and indexing, this is per-objects-field compression.
> > Per-fields compression means that metadata (a header) of an object won't
> > be compressed, only serialized values of an object fields (in bytes array
> > form) will be compressed.
> >
> > This solution have some contentious issues:
> > - small values, like primitives and short arrays - there isn't sense to
> > compress them;
> > - there is no possible to use compression with java-predefined types;
> >
> > We can provide an annotation, @IgniteCompression - for example, which can
> > be used by users for marking fields to compress.
> >
> > Any thoughts?
> >
> > Maybe someone already have ready design?
> >
> > 2017-04-10 11:06 GMT+03:00 Vyacheslav Daradur <daradurvs@gmail.com>:
> >
> >> Alexey,
> >>
> >> Yes, I've read it.
> >>
> >> Ok, let's discuss about public API design.
> >>
> >> I think we need to add some a configure entity to CacheConfiguration,
> >> which will contain the Compressor interface implementation and some
> usefull
> >> parameters.
> >> Or maybe to provide a BinaryMarshaller decorator, which will be compress
> >> data after marshalling.
> >>
> >>
> >> 2017-04-10 10:40 GMT+03:00 Alexey Kuznetsov <akuznetsov@apache.org>:
> >>
> >>> Vyacheslav,
> >>>
> >>> Did you read initial discussion [1] about compression?
> >>> As far as I remember we agreed to add only some "top-level" API in
> order
> >>> to
> >>> provide a way for
> >>> Ignite users to inject some sort of custom compression.
> >>>
> >>>
> >>> [1]
> >>> http://apache-ignite-developers.2346864.n4.nabble.com/Data-c
> >>> ompression-in-Ignite-2-0-td10099.html
> >>>
> >>> On Mon, Apr 10, 2017 at 2:19 PM, daradurvs <daradurvs@gmail.com>
> wrote:
> >>>
> >>> > Hi Igniters!
> >>> >
> >>> > I am interested in this task.
> >>> > Provide some kind of pluggable compression SPI support
> >>> > <https://issues.apache.org/jira/browse/IGNITE-3592>
> >>> >
> >>> > I developed a solution on BinaryMarshaller-level, but reviewer has
> >>> rejected
> >>> > it.
> >>> >
> >>> > Let's continue discussion of task goals and solution design.
> >>> > As I understood that, the main goal of this task is to store data in
> >>> > compressed form.
> >>> > This is what I need from Ignite as its user. Compression provides
> >>> economy
> >>> > on
> >>> > servers.
> >>> > We can store more data on same servers at the cost of increasing CPU
> >>> > utilization.
> >>> >
> >>> > I'm researching a possibility of implementation of compression at the
> >>> > cache-level.
> >>> >
> >>> > Any thoughts?
> >>> >
> >>> > --
> >>> > Best regards,
> >>> > Vyacheslav
> >>> >
> >>> >
> >>> >
> >>> >
> >>> > --
> >>> > View this message in context: http://apache-ignite-
> >>> > developers.2346864.n4.nabble.com/Data-compression-in-
> >>> > Ignite-2-0-tp10099p16317.html
> >>> > Sent from the Apache Ignite Developers mailing list archive at
> >>> Nabble.com.
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Alexey Kuznetsov
> >>>
> >>
> >>
> >>
> >> --
> >> Best Regards, Vyacheslav
> >>
> >
> >
> >
> > --
> > Best Regards, Vyacheslav
> >
>
>
>
> --
> Best Regards, Vyacheslav
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message