ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Ozerov (JIRA)" <j...@apache.org>
Subject [jira] [Created] (IGNITE-6418) Binary: optionally write integer datatypes with varint encoding
Date Mon, 18 Sep 2017 09:28:00 GMT
Vladimir Ozerov created IGNITE-6418:

             Summary: Binary: optionally write integer datatypes with varint encoding
                 Key: IGNITE-6418
                 URL: https://issues.apache.org/jira/browse/IGNITE-6418
             Project: Ignite
          Issue Type: Task
          Components: binary
    Affects Versions: 2.1
            Reporter: Vladimir Ozerov

Currently all integer data types are written as is. {{Integer}} always takes 4 bytes, {{Long}}
- 8 bytes, etc.

There is well-known technique called "varint encoding" which can compress integer values [1].
When used, {{Integer}} can take 1-5 bytes, {{Long}} - 1-10 bytes. So when values are small
enough we can save a lot of space. 

But this technique is not unversal, as big encoded values might require more bytes comparing
to plain form. Also it might cause slowdowns in SQL engine. So this approach cannot be applied
globally. Instead, we should allow users to control whether they want to use this technique
or not.

One possible approach is to add some annotation and several new methods to {{BinaryWriter}}
and {{BinaryReader}}, which will control whether varint is used or not.

[1] https://developers.google.com/protocol-buffers/docs/encoding#varints

This message was sent by Atlassian JIRA

View raw message