Mailing-List: contact dev-help@ignite.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ignite.apache.org
MIME-Version: 1.0
In-Reply-To: <CALH+G9qs+jAr3Q+K+40y_A390ixPfsfZ4PdSNaEhRAznhS+C+A@mail.gmail.com>
References: <CALH+G9qs+jAr3Q+K+40y_A390ixPfsfZ4PdSNaEhRAznhS+C+A@mail.gmail.com>
From: Sergi Vladykin <sergi.vladykin@gmail.com>
Date: Mon, 25 Jul 2016 10:18:13 +0300
Message-ID: <CA+eZwrHRyaLV0--rWZ6uf=ynQPX_KWbSd7=FL8Xaoy4gfa_H_g@mail.gmail.com>
Subject: Re: Data compression in Ignite 2.0
To: dev@ignite.apache.org
Content-Type: multipart/alternative; boundary=94eb2c0480e2b1d5450538709881
archived-at: Mon, 25 Jul 2016 07:18:58 -0000

--94eb2c0480e2b1d5450538709881
Content-Type: text/plain; charset=UTF-8

This will make sense only for rare cases when you have very large objects
stored, which can be effectively compressed. And even then it will
introduce slowdown on all the operations, which often will not be
acceptable. I guess only few users will find this feature useful, thus I
think it does not worth the effort.

Sergi


2016-07-25 9:28 GMT+03:00 Alexey Kuznetsov <akuznetsov@gridgain.com>:

> Hi, All!
>
> I would like to propose one more feature for Ignite 2.0.
>
> Data compression for data in binary format.
>
> Binary format is stored as field name + field data.
> So we have a description.
> How about to add one more byte to binary data descriptor:
>
> *Compressed*:
>  0 - Data stored as is (no compression).
>  1 - Data compressed by dictionary (something like DB2 row compression [1],
>  but for all binary types). We could have system or user defined replicated
> cache for such dictionary and *cache.compact()* method that will scan
> cache, build dictionary and compact data.
>  2 - Data compressed by Java built in ZIP.
>  3 - Data compressed by some user custom algorithm.
>
> Of course it is possible to compress data in current Ignite 1.x but in this
> case compressed data cannot be accessed from SQL engine, if we implement
> support for compression on Ignite core level SQL engine will be able to
> detect that data is compressed and properly handle such data.
>
> What do you think?
> If community consider this feature useful I will create issue in JIRA.
>
> [1]
>
> http://www.ibm.com/developerworks/data/library/techarticle/dm-1205db210compression/
>
> --
> Alexey Kuznetsov
>

--94eb2c0480e2b1d5450538709881--