Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <CAFyTW-h1ZTbTvX6D_KNhOZ7Pb4F0_jWJ11PncFqi2UvQAc=SWg@mail.gmail.com>
References: <CALV17QrUX1DRNP8KYH5Hz5C5Sv7QmFifD7qar1jaZLQQhcPA-A@mail.gmail.com>
 <CA+0=VoUG2-=L2CxnCykzJnM7OY8pYhw5jMietVNdE7+DaPuRCg@mail.gmail.com>
 <DM2PR0701MB1230B43298AEC92D7C1E051BCF0E0@DM2PR0701MB1230.namprd07.prod.outlook.com>
 <CALH+G9pg5t+Z5NNpaSTpiBx927PZzRjgey=+G4+KxMiFmHBcSA@mail.gmail.com>
 <DM2PR0701MB1230BB9221156D6F1318CA33CF0F0@DM2PR0701MB1230.namprd07.prod.outlook.com>
 <CALV17QqBivBpJkDnsHfT4j+p+oWDQC5SHV0eGZDx+Hu=qeYuBQ@mail.gmail.com>
 <CALH+G9ri=TuQ32bBggTMbeqxwtr8+O9fS4MznGMkAZKwrNh=Bg@mail.gmail.com>
 <CAGcYxn0tgL8ZhQSMfx3KJw9Zg_+uKv1bU=3Z11ukmn1Rhxa=wA@mail.gmail.com>
 <CA+eZwrFSuraDEZdx9f3gZcmA_DDn2Pzj_FD3GB9xNxWAEpQUDA@mail.gmail.com>
 <CALH+G9q_eVxZevzf487snPoBjVBsnHZONE3zKzusdx5w6ix=zA@mail.gmail.com>
 <1491808789290-16317.post@n4.nabble.com> <CALH+G9qR0X4EymHQS+SaOgD0oO9qDFbVW7mNvNzi-iRfJojAHg@mail.gmail.com>
 <CAFyTW-g1OgCwpHhsvx5MZsw-gssWxqcO-vUzmwKGE1Kujir2yw@mail.gmail.com> <CAFyTW-h1ZTbTvX6D_KNhOZ7Pb4F0_jWJ11PncFqi2UvQAc=SWg@mail.gmail.com>
From: Vyacheslav Daradur <daradurvs@gmail.com>
Date: Mon, 15 May 2017 16:36:07 +0300
Message-ID: <CAFyTW-giRDfpL_F2n0eEb2LOcrqHxfo1o_km-JKcmt4w37mUKw@mail.gmail.com>
Subject: Re: Data compression in Ignite 2.0
To: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary="001a11c002162df1b8054f90232b"
archived-at: Mon, 15 May 2017 13:36:13 -0000

--001a11c002162df1b8054f90232b
Content-Type: text/plain; charset="UTF-8"

I created the ticket: https://issues.apache.org/jira/browse/IGNITE-5226

I'll prepare a PR with described solution in couple of days.

2017-05-15 15:05 GMT+03:00 Vyacheslav Daradur <daradurvs@gmail.com>:

> Hi, Igniters!
>
> Apache 2.0 is released.
>
> Let's continue the discussion about a compression design.
>
> At the moment, I found only one solution which is compatible with querying
> and indexing, this is per-objects-field compression.
> Per-fields compression means that metadata (a header) of an object won't
> be compressed, only serialized values of an object fields (in bytes array
> form) will be compressed.
>
> This solution have some contentious issues:
> - small values, like primitives and short arrays - there isn't sense to
> compress them;
> - there is no possible to use compression with java-predefined types;
>
> We can provide an annotation, @IgniteCompression - for example, which can
> be used by users for marking fields to compress.
>
> Any thoughts?
>
> Maybe someone already have ready design?
>
> 2017-04-10 11:06 GMT+03:00 Vyacheslav Daradur <daradurvs@gmail.com>:
>
>> Alexey,
>>
>> Yes, I've read it.
>>
>> Ok, let's discuss about public API design.
>>
>> I think we need to add some a configure entity to CacheConfiguration,
>> which will contain the Compressor interface implementation and some usefull
>> parameters.
>> Or maybe to provide a BinaryMarshaller decorator, which will be compress
>> data after marshalling.
>>
>>
>> 2017-04-10 10:40 GMT+03:00 Alexey Kuznetsov <akuznetsov@apache.org>:
>>
>>> Vyacheslav,
>>>
>>> Did you read initial discussion [1] about compression?
>>> As far as I remember we agreed to add only some "top-level" API in order
>>> to
>>> provide a way for
>>> Ignite users to inject some sort of custom compression.
>>>
>>>
>>> [1]
>>> http://apache-ignite-developers.2346864.n4.nabble.com/Data-c
>>> ompression-in-Ignite-2-0-td10099.html
>>>
>>> On Mon, Apr 10, 2017 at 2:19 PM, daradurvs <daradurvs@gmail.com> wrote:
>>>
>>> > Hi Igniters!
>>> >
>>> > I am interested in this task.
>>> > Provide some kind of pluggable compression SPI support
>>> > <https://issues.apache.org/jira/browse/IGNITE-3592>
>>> >
>>> > I developed a solution on BinaryMarshaller-level, but reviewer has
>>> rejected
>>> > it.
>>> >
>>> > Let's continue discussion of task goals and solution design.
>>> > As I understood that, the main goal of this task is to store data in
>>> > compressed form.
>>> > This is what I need from Ignite as its user. Compression provides
>>> economy
>>> > on
>>> > servers.
>>> > We can store more data on same servers at the cost of increasing CPU
>>> > utilization.
>>> >
>>> > I'm researching a possibility of implementation of compression at the
>>> > cache-level.
>>> >
>>> > Any thoughts?
>>> >
>>> > --
>>> > Best regards,
>>> > Vyacheslav
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > View this message in context: http://apache-ignite-
>>> > developers.2346864.n4.nabble.com/Data-compression-in-
>>> > Ignite-2-0-tp10099p16317.html
>>> > Sent from the Apache Ignite Developers mailing list archive at
>>> Nabble.com.
>>> >
>>>
>>>
>>>
>>> --
>>> Alexey Kuznetsov
>>>
>>
>>
>>
>> --
>> Best Regards, Vyacheslav
>>
>
>
>
> --
> Best Regards, Vyacheslav
>


-- 
Best Regards, Vyacheslav

--001a11c002162df1b8054f90232b--