ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Kuznetsov <akuznet...@gridgain.com>
Subject Data compression in Ignite 2.0
Date Mon, 25 Jul 2016 06:28:26 GMT
Hi, All!

I would like to propose one more feature for Ignite 2.0.

Data compression for data in binary format.

Binary format is stored as field name + field data.
So we have a description.
How about to add one more byte to binary data descriptor:

*Compressed*:
 0 - Data stored as is (no compression).
 1 - Data compressed by dictionary (something like DB2 row compression [1],
 but for all binary types). We could have system or user defined replicated
cache for such dictionary and *cache.compact()* method that will scan
cache, build dictionary and compact data.
 2 - Data compressed by Java built in ZIP.
 3 - Data compressed by some user custom algorithm.

Of course it is possible to compress data in current Ignite 1.x but in this
case compressed data cannot be accessed from SQL engine, if we implement
support for compression on Ignite core level SQL engine will be able to
detect that data is compressed and properly handle such data.

What do you think?
If community consider this feature useful I will create issue in JIRA.

[1]
http://www.ibm.com/developerworks/data/library/techarticle/dm-1205db210compression/

-- 
Alexey Kuznetsov

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message