ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sergi Vladykin <sergi.vlady...@gmail.com>
Subject Re: Data compression in Ignite 2.0
Date Mon, 25 Jul 2016 07:18:13 GMT
This will make sense only for rare cases when you have very large objects
stored, which can be effectively compressed. And even then it will
introduce slowdown on all the operations, which often will not be
acceptable. I guess only few users will find this feature useful, thus I
think it does not worth the effort.

Sergi



2016-07-25 9:28 GMT+03:00 Alexey Kuznetsov <akuznetsov@gridgain.com>:

> Hi, All!
>
> I would like to propose one more feature for Ignite 2.0.
>
> Data compression for data in binary format.
>
> Binary format is stored as field name + field data.
> So we have a description.
> How about to add one more byte to binary data descriptor:
>
> *Compressed*:
>  0 - Data stored as is (no compression).
>  1 - Data compressed by dictionary (something like DB2 row compression [1],
>  but for all binary types). We could have system or user defined replicated
> cache for such dictionary and *cache.compact()* method that will scan
> cache, build dictionary and compact data.
>  2 - Data compressed by Java built in ZIP.
>  3 - Data compressed by some user custom algorithm.
>
> Of course it is possible to compress data in current Ignite 1.x but in this
> case compressed data cannot be accessed from SQL engine, if we implement
> support for compression on Ignite core level SQL engine will be able to
> detect that data is compressed and properly handle such data.
>
> What do you think?
> If community consider this feature useful I will create issue in JIRA.
>
> [1]
>
> http://www.ibm.com/developerworks/data/library/techarticle/dm-1205db210compression/
>
> --
> Alexey Kuznetsov
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message