ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: Data compression in Ignite 2.0
Date Fri, 09 Jun 2017 07:50:52 GMT
Dima,

Encryption of certain fields is as bad as compression. First, it is a huge
change, which makes already complex binary protocol even more complex.
Second, it have to be ported to CPP, .NET platforms, as well as to JDBC and
ODBC.
Last, but the most important - this is not our headache to encrypt
sensitive data. This is user responsibility. Nobody in a sane mind will
store passwords in plain form. Instead, user should encrypt it on his own,
choosing proper encryption parameters - algorithms, key lengths, salts,
etc.. How are you going to expose this in API or configuration?

We should not implement data encryption on binary level, this is out of
question. Encryption should be implemented on application level (user
efforts), transport layer (SSL - we already have it), and possibly on
disk-level (there are tools for this already).


On Fri, Jun 9, 2017 at 9:06 AM, Vyacheslav Daradur <daradurvs@gmail.com>
wrote:

> >> which is much less useful.
> I note, in some cases there is profit more than twice per size of an
> object.
>
> >> Would it be possible to change your implementation to handle the
> encryption instead?
> Yes, of cource, there's not much difference between compression and
> encryption, including in my implementation of per-field-compression.
>
> 2017-06-09 8:55 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:
>
> > Vyacheslav,
> >
> > When this feature started out as data compression in Ignite, it sounded
> > very useful. Now it is unfolding as a per-field compression, which is
> much
> > less useful. In fact, it is questionable whether it is useful at all. The
> > fact that this feature is implemented does not make it mandatory for the
> > community to accept it.
> >
> > However, as I mentioned before, per-field encryption is very useful, as
> it
> > would allow users automatically encrypt certain sensitive fields, like
> > passwords, credit card numbers, etc. There is not much conceptual
> > difference between compressing a field vs encrypting a field. Would it be
> > possible to change your implementation to handle the encryption instead?
> >
> > D.
> >
> > On Thu, Jun 8, 2017 at 10:42 PM, Vyacheslav Daradur <daradurvs@gmail.com
> >
> > wrote:
> >
> > > Guys, I want to be clear:
> > > * "Per-field compression" design is the result of a research of the
> > binary
> > > infrastructure of Ignite and some other its places (querying, indexing,
> > > etc.)
> > > * Full-compression of object will be more effective, but in this case
> > there
> > > is no capability with querying and indexing (or there is large overhead
> > by
> > > way of decompressing of full object (or caches pages) on demand)
> > > * "Per-field compression" is a one of ways to implement the compression
> > > feature
> > >
> > > I'm new to Ignite also I can be mistaken in some things.
> > > Last 3-4 month I've tryed to start dicussion about a design, but nobody
> > > answers nothing (except Dmitry and Valentin who was interested how it
> > > works).
> > > But I understand that this is community and nobody is obliged to
> anybody.
> > >
> > > There are strong Ignite experts.
> > > If they can help me and community with a design of the compression
> > feature
> > > it will be great.
> > > At the moment I have a desire and time to be engaged in development of
> > > compression feature in Ignite.
> > > Let's use this opportunity :)
> > >
> > > 2017-06-09 5:36 GMT+03:00 Dmitriy Setrakyan <dsetrakyan@apache.org>:
> > >
> > > > Igniters,
> > > >
> > > > I have never seen a single Ignite user asking about compressing a
> > single
> > > > field. However, we have had requests to secure certain fields, e.g.
> > > > passwords.
> > > >
> > > > I personally do not think per-field compression is needed, unless we
> > can
> > > > point out some concrete real life use cases.
> > > >
> > > > D.
> > > >
> > > > On Thu, Jun 8, 2017 at 3:42 AM, Vyacheslav Daradur <
> > daradurvs@gmail.com>
> > > > wrote:
> > > >
> > > > > Anton,
> > > > >
> > > > > >> I thought that if there will storing compressed data in the
> > memory,
> > > > data
> > > > > >> will transmit over wire in compression too. Is it right?
> > > > >
> > > > > In per-field compression case - yes.
> > > > >
> > > > > 2017-06-08 13:36 GMT+03:00 Антон Чураев <churaev.an@gmail.com>:
> > > > >
> > > > > > Guys, could you please help me.
> > > > > > I thought that if there will storing compressed data in the
> memory,
> > > > data
> > > > > > will transmit over wire in compression too. Is it right?
> > > > > >
> > > > > > 2017-06-08 13:30 GMT+03:00 Vyacheslav Daradur <
> daradurvs@gmail.com
> > >:
> > > > > >
> > > > > > > Vladimir,
> > > > > > >
> > > > > > > The main problem which I'am trying to solve is storing data in
> > > memory
> > > > > in
> > > > > > a
> > > > > > > compression form via Ignite.
> > > > > > > The main goal is using memory more effectivelly.
> > > > > > >
> > > > > > > >> here the much simpler step would be to full
> > > > > > > compression on per-cache basis rather than dealing with
> > per-fields
> > > > > case.
> > > > > > >
> > > > > > > Please explain your idea. Compess data by memory-page?
> > > > > > > Is it compatible with quering and indexing?
> > > > > > >
> > > > > > > >> In the end, if user would like to compress particular field,
> > he
> > > > can
> > > > > > > always to it on his own
> > > > > > > I think we mustn't think in this way, if user need something he
> > > > trying
> > > > > to
> > > > > > > choose a tool which has this feature OOTB.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > 2017-06-08 12:53 GMT+03:00 Vladimir Ozerov <
> vozerov@gridgain.com
> > >:
> > > > > > >
> > > > > > > > Igniters,
> > > > > > > >
> > > > > > > > Honestly I still do not see how to apply it gracefully this
> > > feature
> > > > > ti
> > > > > > > > Ignite. And overall approach to compress only particular
> fields
> > > > looks
> > > > > > > > overcomplicated to me. Remember, that our main use case is an
> > > > > > application
> > > > > > > > without classes on the server. It means that any kind of
> > > > annotations
> > > > > > are
> > > > > > > > inapplicable. To be more precise: proper API should be
> > > implemented
> > > > to
> > > > > > > > handle no-class case (e.g. how would build such an object
> > through
> > > > > > > > BinaryBuilder without a class?), and only then add
> annotations
> > as
> > > > > > > > convenient addition to more basic API.
> > > > > > > >
> > > > > > > > It seems to me that full implementation, which takes in count
> > > > proper
> > > > > > > > "classless" API, changes to binary metadata to reflect
> > compressed
> > > > > > fields,
> > > > > > > > changes to SQL, changes to binary protocol, and porting to
> .NET
> > > and
> > > > > > CPP,
> > > > > > > > will yield very complex solution with little value to the
> > > product.
> > > > > > > >
> > > > > > > > Instead, as I proposed earlier, it seems that we'd better
> start
> > > > with
> > > > > > the
> > > > > > > > problem we are trying to solve. Basically, compression could
> > help
> > > > in
> > > > > > two
> > > > > > > > cases:
> > > > > > > > 1) Transmitting data over wire - it should be implemented on
> > > > > > > communication
> > > > > > > > layer and should not affect binary serialization component a
> > lot.
> > > > > > > > 2) Storing data in memory - here the much simpler step would
> be
> > > to
> > > > > full
> > > > > > > > compression on per-cache basis rather than dealing with
> > > per-fields
> > > > > > case.
> > > > > > > >
> > > > > > > > In the end, if user would like to compress particular field,
> he
> > > can
> > > > > > > always
> > > > > > > > to it on his own, and set already compressed field to our
> > > > > BinaryObject.
> > > > > > > >
> > > > > > > > Vladimir.
> > > > > > > >
> > > > > > > >
> > > > > > > > On Thu, Jun 8, 2017 at 12:37 PM, Vyacheslav Daradur <
> > > > > > daradurvs@gmail.com
> > > > > > > >
> > > > > > > > wrote:
> > > > > > > >
> > > > > > > > > Valentin,
> > > > > > > > >
> > > > > > > > > Yes, I have the prototype[1][2]
> > > > > > > > >
> > > > > > > > > You can see an example of Java class[3] that I used in my
> > > > > benchmark.
> > > > > > > > > For example:
> > > > > > > > > class Foo {
> > > > > > > > > @BinaryCompression
> > > > > > > > > String data;
> > > > > > > > > }
> > > > > > > > > If user make decision to store the object in compressed
> form,
> > > he
> > > > > can
> > > > > > > use
> > > > > > > > > the annotation @BinaryCompression as shown above.
> > > > > > > > > It means annotated field 'data' will be compressed at
> > > > marshalling.
> > > > > > > > >
> > > > > > > > > [1] https://github.com/apache/ignite/pull/1951
> > > > > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-5226
> > > > > > > > > [3]
> > > > > > > > > https://github.com/daradurvs/ignite-compression/blob/
> > > > > > > > > master/src/main/java/ru/daradurvs/ignite/compression/
> > > > > > > model/Audit1F.java
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > 2017-06-08 2:04 GMT+03:00 Valentin Kulichenko <
> > > > > > > > > valentin.kulichenko@gmail.com
> > > > > > > > > >:
> > > > > > > > >
> > > > > > > > > > Vyacheslav, Anton,
> > > > > > > > > >
> > > > > > > > > > Are there any ideas and/or prototypes for the API? Your
> > > design
> > > > > > > > > suggestions
> > > > > > > > > > seem to make sense, but I would like to see how it all
> this
> > > > will
> > > > > > like
> > > > > > > > > from
> > > > > > > > > > user's standpoint.
> > > > > > > > > >
> > > > > > > > > > -Val
> > > > > > > > > >
> > > > > > > > > > On Wed, Jun 7, 2017 at 1:06 AM, Антон Чураев <
> > > > > churaev.an@gmail.com
> > > > > > >
> > > > > > > > > wrote:
> > > > > > > > > >
> > > > > > > > > > > Vyacheslav, correct me if something wrong
> > > > > > > > > > >
> > > > > > > > > > > We could provide opportunity of choose between CPU
> usage
> > > and
> > > > > > > MEM/NET
> > > > > > > > > > usage
> > > > > > > > > > > for users by compression some attributes of stored
> > objects.
> > > > > > > > > > > You have learned design, and it is possible to localize
> > > > changes
> > > > > > in
> > > > > > > > > > > marshalling without performance affect and current
> > > > > functionality.
> > > > > > > > > > >
> > > > > > > > > > > I think, that it's usefull for our project and users.
> > > > > > > > > > > Community, what do you think about this proposal?
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > 2017-06-06 17:29 GMT+03:00 Vyacheslav Daradur <
> > > > > > daradurvs@gmail.com
> > > > > > > >:
> > > > > > > > > > >
> > > > > > > > > > > > In short,
> > > > > > > > > > > >
> > > > > > > > > > > > During marshalling a fields is represented as
> > > > > > BinaryFieldAccessor
> > > > > > > > > which
> > > > > > > > > > > > manages its marshalling. It checks if the field is
> > marked
> > > > by
> > > > > > > > > annotation
> > > > > > > > > > > > @BinaryCompression, in that case - binary
> > representation
> > > > of
> > > > > > > field
> > > > > > > > > > (bytes
> > > > > > > > > > > > array) will be compressed. It will be marked as
> > > compressed
> > > > by
> > > > > > > types
> > > > > > > > > > > > constant (GridBinaryMarshaller.COMPRESSED), after
> this
> > > the
> > > > > > > > > compressed
> > > > > > > > > > > > bytes
> > > > > > > > > > > > array wiil be include in binary representation of
> whole
> > > > > object.
> > > > > > > > Note,
> > > > > > > > > > > > header of marshalled object will not be compressed.
> > > > > Compression
> > > > > > > > > > affected
> > > > > > > > > > > > only object's field representation.
> > > > > > > > > > > >
> > > > > > > > > > > > Objects in IgniteCache is represented as BinaryObject
> > > which
> > > > > is
> > > > > > > > > wrapper
> > > > > > > > > > > over
> > > > > > > > > > > > bytes array of marshalled object.
> > > > > > > > > > > > BinaryObject provides some usefull methods, which are
> > > used
> > > > by
> > > > > > > > Ignite
> > > > > > > > > > > > systems.
> > > > > > > > > > > > For example, the Queries use BinaryObject#field
> method,
> > > > which
> > > > > > > > > > > deserializes
> > > > > > > > > > > > only field of object, without deserializing of whole
> > > > object.
> > > > > > > > > > > > BinaryObject#field method during deserialization, if
> > > meets
> > > > > the
> > > > > > > > > constant
> > > > > > > > > > > of
> > > > > > > > > > > > compressed type, decompress this bytes array, then
> > > continue
> > > > > > > > > > unmarshalling
> > > > > > > > > > > > as usual.
> > > > > > > > > > > >
> > > > > > > > > > > > Now, I introduced the Compressor interface in
> > > > > > > IgniteConfigurations,
> > > > > > > > > it
> > > > > > > > > > > > allows user to use own implementation of compressor -
> > it
> > > is
> > > > > the
> > > > > > > > > > > requirement
> > > > > > > > > > > > in the task[1].
> > > > > > > > > > > >
> > > > > > > > > > > > As far as I know, Vladimir Ozerov doesn't like the
> idea
> > > of
> > > > > > > granting
> > > > > > > > > > this
> > > > > > > > > > > > opportunity to the user.
> > > > > > > > > > > > In that case we can choose a compression algorithm
> > which
> > > we
> > > > > > will
> > > > > > > > > > provide
> > > > > > > > > > > by
> > > > > > > > > > > > default and will move the interface to internals of
> > > binary
> > > > > > > > > > > infractructure.
> > > > > > > > > > > > For this case I've prepared benchmarked, which I've
> > sent
> > > > > > earlier.
> > > > > > > > > > > >
> > > > > > > > > > > > I vote for ZSTD algorithm[2], it provides good
> > > compression
> > > > > > ratio
> > > > > > > > and
> > > > > > > > > > good
> > > > > > > > > > > > throughput. It has implementation in Java, .NET and
> > C++,
> > > > and
> > > > > > has
> > > > > > > > > > > > ASF-friendly license, we can use it in the all Ignite
> > > > > > platforms.
> > > > > > > > > > > > You can look at an assessment of this algorithm in my
> > > > > > benchmark's
> > > > > > > > > > > >
> > > > > > > > > > > > [1] https://issues.apache.org/
> jira/browse/IGNITE-3592
> > > > > > > > > > > > [2]https://github.com/facebook/zstd
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > 2017-06-06 16:02 GMT+03:00 Антон Чураев <
> > > > > churaev.an@gmail.com
> > > > > > >:
> > > > > > > > > > > >
> > > > > > > > > > > > > Looks good for me.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Could You propose design of implementation in
> couple
> > of
> > > > > > > > sentences?
> > > > > > > > > > > > > So that we can estimate the completeness and
> > complexity
> > > > of
> > > > > > the
> > > > > > > > > > > proposal.
> > > > > > > > > > > > >
> > > > > > > > > > > > > 2017-06-06 15:26 GMT+03:00 Vyacheslav Daradur <
> > > > > > > > daradurvs@gmail.com
> > > > > > > > > >:
> > > > > > > > > > > > >
> > > > > > > > > > > > > > Anton,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Of course, the solution does not affect on
> existing
> > > > > > > > > > implementation. I
> > > > > > > > > > > > > mean,
> > > > > > > > > > > > > > there is no changes if user not use the
> annotation
> > > > > > > > > > > @BinaryCompression.
> > > > > > > > > > > > > (no
> > > > > > > > > > > > > > performance changes)
> > > > > > > > > > > > > > Only if user make decision to use compression on
> > > > specific
> > > > > > > field
> > > > > > > > > or
> > > > > > > > > > > > fields
> > > > > > > > > > > > > > of a class - in that case compression will be
> used
> > at
> > > > > > > > marshalling
> > > > > > > > > > in
> > > > > > > > > > > > > > relation to annotated fields.
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > 2017-06-06 15:10 GMT+03:00 Антон Чураев <
> > > > > > > churaev.an@gmail.com
> > > > > > > > >:
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Vyacheslav,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Is it possible to propose implementation that
> can
> > > be
> > > > > > > switched
> > > > > > > > > on
> > > > > > > > > > > > > > on-demand?
> > > > > > > > > > > > > > > In this case it should not affect performance
> of
> > > > > current
> > > > > > > > > > solution.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I mean, that users should make decision what is
> > > more
> > > > > > > > important
> > > > > > > > > > for
> > > > > > > > > > > > > them:
> > > > > > > > > > > > > > > throutput or memory/net usage.
> > > > > > > > > > > > > > > May be they will be choose not all objects, or
> > only
> > > > > some
> > > > > > > > > > attributes
> > > > > > > > > > > > of
> > > > > > > > > > > > > > > objects for compress.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > 2017-06-06 14:48 GMT+03:00 Vyacheslav Daradur <
> > > > > > > > > > daradurvs@gmail.com
> > > > > > > > > > > >:
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Conclusion:
> > > > > > > > > > > > > > > > Provided solution allows reduce size of an
> > object
> > > > in
> > > > > > > > > > IgniteCache
> > > > > > > > > > > at
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > cost of throughput reduction (small - in some
> > > > cases),
> > > > > > it
> > > > > > > > > > depends
> > > > > > > > > > > on
> > > > > > > > > > > > > > part
> > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > object which will be compressed and
> compression
> > > > > > > algorithm.
> > > > > > > > > > > > > > > > I mean, we can make more effective use of
> > memory,
> > > > and
> > > > > > in
> > > > > > > > some
> > > > > > > > > > > cases
> > > > > > > > > > > > > it
> > > > > > > > > > > > > > > can
> > > > > > > > > > > > > > > > reduce loading of the interconnect.
> > (replication,
> > > > > > > > > rebalancing)
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Especially, it will be particularly useful
> for
> > > > > object's
> > > > > > > > > fields
> > > > > > > > > > > > which
> > > > > > > > > > > > > > are
> > > > > > > > > > > > > > > > large text (>~ 250 bytes) and can be
> > effectively
> > > > > > > > compressed.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > 2017-06-06 12:00 GMT+03:00 Антон Чураев <
> > > > > > > > > churaev.an@gmail.com
> > > > > > > > > > >:
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Vyacheslav, thank you! But could you please
> > > > > provide a
> > > > > > > > > > > conclusions
> > > > > > > > > > > > > or
> > > > > > > > > > > > > > > > > proposals based on this benchmarks?
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > 2017-06-06 11:28 GMT+03:00 Vyacheslav
> > Daradur <
> > > > > > > > > > > > daradurvs@gmail.com
> > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Dmitry,
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > Excel-pages:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > 1). "Compression ratio (2)" - shows
> object
> > > > size,
> > > > > > with
> > > > > > > > > > > > compression
> > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > without compression. (Conditions: literal
> > > text)
> > > > > > > > > > > > > > > > > > 1st graph shows compression ratios of
> using
> > > > > > different
> > > > > > > > > > > > compression
> > > > > > > > > > > > > > > > > algrithms
> > > > > > > > > > > > > > > > > > depending on size of compressed field.
> > > > > > > > > > > > > > > > > > 2nd graph shows evaluation of size of
> > objects
> > > > > > > depending
> > > > > > > > > on
> > > > > > > > > > > > sizes
> > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > compression algorithms.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > 2). "Compression ratio (1)" - shows
> object
> > > > size,
> > > > > > with
> > > > > > > > > > > > compression
> > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > without compression. (Conditions:  badly
> > > > > compressed
> > > > > > > > > > character
> > > > > > > > > > > > > > > sequence)
> > > > > > > > > > > > > > > > > > 1st graph shows compression ratios of
> using
> > > > > > different
> > > > > > > > > > > > compression
> > > > > > > > > > > > > > > > > > algrithms depending on size of compressed
> > > > field.
> > > > > > > > > > > > > > > > > > 2nd graph shows evaluation of size of
> > objects
> > > > > > > depending
> > > > > > > > > on
> > > > > > > > > > > > sizes
> > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > compression algorithms.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > 3) 'put-avg" - shows average time of the
> > > "put"
> > > > > > > > operation
> > > > > > > > > > > > > depending
> > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > size
> > > > > > > > > > > > > > > > > > and compression algorithms.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > 4) 'put-thrpt" - shows throughput of the
> > > "put"
> > > > > > > > operation
> > > > > > > > > > > > > depending
> > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > size
> > > > > > > > > > > > > > > > > > and compression algorithms.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > 5) 'get-avg" - shows average time of the
> > > "get"
> > > > > > > > operation
> > > > > > > > > > > > > depending
> > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > size
> > > > > > > > > > > > > > > > > > and compression algorithms.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > 6) 'get-thrpt" - shows throughput of the
> > > "get"
> > > > > > > > operation
> > > > > > > > > > > > > depending
> > > > > > > > > > > > > > on
> > > > > > > > > > > > > > > > > size
> > > > > > > > > > > > > > > > > > and compression algorithms.
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > 2017-06-06 10:59 GMT+03:00 Dmitriy
> > Setrakyan
> > > <
> > > > > > > > > > > > > > dsetrakyan@apache.org
> > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > Vladimir, I am not sure how to
> interpret
> > > the
> > > > > > > graphs?
> > > > > > > > > What
> > > > > > > > > > > are
> > > > > > > > > > > > > we
> > > > > > > > > > > > > > > > > looking
> > > > > > > > > > > > > > > > > > > at?
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > On Tue, Jun 6, 2017 at 12:33 AM,
> > Vyacheslav
> > > > > > > Daradur <
> > > > > > > > > > > > > > > > > daradurvs@gmail.com
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > wrote:
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > Hi, Igniters.
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > I've prepared some benchmarking.
> > Results
> > > > [1].
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > And I've prepared the evaluation in
> the
> > > > form
> > > > > of
> > > > > > > > > > diagrams
> > > > > > > > > > > > [2].
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > I hope that helps to interest the
> > > community
> > > > > and
> > > > > > > > > > > > accelerates a
> > > > > > > > > > > > > > > > > reaction
> > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > this improvment :)
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > [1]
> > > > > > > > > > > > > > > > > > > > https://github.com/daradurvs/
> > > > > > > > > ignite-compression/tree/
> > > > > > > > > > > > > > > > > > > > master/src/main/resources/result
> > > > > > > > > > > > > > > > > > > > [2] https://drive.google.com/file/d/
> > > > > > > > > > > > > > > 0B2CeUAOgrHkoMklyZ25YTEdKcEk/
> > > > > > > > > > > > > > > > > view
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > 2017-05-24 9:49 GMT+03:00 Vyacheslav
> > > > Daradur
> > > > > <
> > > > > > > > > > > > > > > daradurvs@gmail.com
> > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > Guys, any thoughts?
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > 2017-05-16 13:40 GMT+03:00
> Vyacheslav
> > > > > > Daradur <
> > > > > > > > > > > > > > > > daradurvs@gmail.com
> > > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >> Hi guys,
> > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > >> I've prepared the PR to show my
> > idea.
> > > > > > > > > > > > > > > > > > > > >> https://github.com/apache/
> > > > > > > > ignite/pull/1951/files
> > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > >> About querying - I've just copied
> > > > existing
> > > > > > > tests
> > > > > > > > > and
> > > > > > > > > > > > have
> > > > > > > > > > > > > > > > > annotated
> > > > > > > > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > > > >> testing data.
> > > > > > > > > > > > > > > > > > > > >> https://github.com/apache/
> > > > > > > > > > > ignite/pull/1951/files#diff-
> > > > > > > > > > > > > > c19a9d
> > > > > > > > > > > > > > > > > > > > >> f4058141d059bb577e75244764
> > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > >> It means fields which will be
> marked
> > > by
> > > > > > > > > > > > @BinaryCompression
> > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > > > > > >> compressed at marshalling via
> > > > > > > BinaryMarshaller.
> > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > >> This solution has no effect on
> > > existing
> > > > > data
> > > > > > > or
> > > > > > > > > > > project
> > > > > > > > > > > > > > > > > > architecture.
> > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > >> I'll be glad to see your thougths.
> > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > >> 2017-05-15 19:18 GMT+03:00
> > Vyacheslav
> > > > > > Daradur
> > > > > > > <
> > > > > > > > > > > > > > > > > daradurvs@gmail.com
> > > > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > >>> Dmitriy,
> > > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > > >>> I have ready prototype. I want to
> > > show
> > > > > it.
> > > > > > > > > > > > > > > > > > > > >>> It is always easier to discuss on
> > > > > example.
> > > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > > >>> 2017-05-15 19:02 GMT+03:00
> Dmitriy
> > > > > > Setrakyan
> > > > > > > <
> > > > > > > > > > > > > > > > > > dsetrakyan@apache.org
> > > > > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > > >>>> Vyacheslav,
> > > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > > >>>> I think it is a bit premature to
> > > > > provide a
> > > > > > > PR
> > > > > > > > > > > without
> > > > > > > > > > > > > > > getting
> > > > > > > > > > > > > > > > a
> > > > > > > > > > > > > > > > > > > > >>>> community
> > > > > > > > > > > > > > > > > > > > >>>> consensus on the dev list.
> Please
> > > > allow
> > > > > > some
> > > > > > > > > time
> > > > > > > > > > > for
> > > > > > > > > > > > > the
> > > > > > > > > > > > > > > > > > community
> > > > > > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > >>>> respond.
> > > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > > >>>> D.
> > > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > > >>>> On Mon, May 15, 2017 at 6:36 AM,
> > > > > > Vyacheslav
> > > > > > > > > > Daradur
> > > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > > >>>> daradurvs@gmail.com>
> > > > > > > > > > > > > > > > > > > > >>>> wrote:
> > > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > > >>>> > I created the ticket:
> > > > > > > > > > > > https://issues.apache.org/jira
> > > > > > > > > > > > > > > > > > > > >>>> /browse/IGNITE-5226
> > > > > > > > > > > > > > > > > > > > >>>> >
> > > > > > > > > > > > > > > > > > > > >>>> > I'll prepare a PR with
> described
> > > > > > solution
> > > > > > > in
> > > > > > > > > > > couple
> > > > > > > > > > > > of
> > > > > > > > > > > > > > > days.
> > > > > > > > > > > > > > > > > > > > >>>> >
> > > > > > > > > > > > > > > > > > > > >>>> > 2017-05-15 15:05 GMT+03:00
> > > > Vyacheslav
> > > > > > > > Daradur
> > > > > > > > > <
> > > > > > > > > > > > > > > > > > > daradurvs@gmail.com
> > > > > > > > > > > > > > > > > > > > >:
> > > > > > > > > > > > > > > > > > > > >>>> >
> > > > > > > > > > > > > > > > > > > > >>>> > > Hi, Igniters!
> > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > >>>> > > Apache 2.0 is released.
> > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > >>>> > > Let's continue the
> discussion
> > > > about
> > > > > a
> > > > > > > > > > > compression
> > > > > > > > > > > > > > > design.
> > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > >>>> > > At the moment, I found only
> > one
> > > > > > solution
> > > > > > > > > which
> > > > > > > > > > > is
> > > > > > > > > > > > > > > > compatible
> > > > > > > > > > > > > > > > > > > with
> > > > > > > > > > > > > > > > > > > > >>>> > querying
> > > > > > > > > > > > > > > > > > > > >>>> > > and indexing, this is
> > > > > > per-objects-field
> > > > > > > > > > > > compression.
> > > > > > > > > > > > > > > > > > > > >>>> > > Per-fields compression means
> > > that
> > > > > > > metadata
> > > > > > > > > (a
> > > > > > > > > > > > > header)
> > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > an
> > > > > > > > > > > > > > > > > > > object
> > > > > > > > > > > > > > > > > > > > >>>> won't
> > > > > > > > > > > > > > > > > > > > >>>> > > be compressed, only
> serialized
> > > > > values
> > > > > > of
> > > > > > > > an
> > > > > > > > > > > object
> > > > > > > > > > > > > > > fields
> > > > > > > > > > > > > > > > > (in
> > > > > > > > > > > > > > > > > > > > bytes
> > > > > > > > > > > > > > > > > > > > >>>> array
> > > > > > > > > > > > > > > > > > > > >>>> > > form) will be compressed.
> > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > >>>> > > This solution have some
> > > > contentious
> > > > > > > > issues:
> > > > > > > > > > > > > > > > > > > > >>>> > > - small values, like
> > primitives
> > > > and
> > > > > > > short
> > > > > > > > > > > arrays -
> > > > > > > > > > > > > > there
> > > > > > > > > > > > > > > > > isn't
> > > > > > > > > > > > > > > > > > > > >>>> sense to
> > > > > > > > > > > > > > > > > > > > >>>> > > compress them;
> > > > > > > > > > > > > > > > > > > > >>>> > > - there is no possible to
> use
> > > > > > > compression
> > > > > > > > > with
> > > > > > > > > > > > > > > > > java-predefined
> > > > > > > > > > > > > > > > > > > > >>>> types;
> > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > >>>> > > We can provide an
> annotation,
> > > > > > > > > > > @IgniteCompression -
> > > > > > > > > > > > > for
> > > > > > > > > > > > > > > > > > example,
> > > > > > > > > > > > > > > > > > > > >>>> which can
> > > > > > > > > > > > > > > > > > > > >>>> > > be used by users for marking
> > > > fields
> > > > > to
> > > > > > > > > > compress.
> > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > >>>> > > Any thoughts?
> > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > >>>> > > Maybe someone already have
> > ready
> > > > > > design?
> > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > >>>> > > 2017-04-10 11:06 GMT+03:00
> > > > > Vyacheslav
> > > > > > > > > Daradur
> > > > > > > > > > <
> > > > > > > > > > > > > > > > > > > > daradurvs@gmail.com
> > > > > > > > > > > > > > > > > > > > >>>> >:
> > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > >>>> > >> Alexey,
> > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > >>>> > >> Yes, I've read it.
> > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > >>>> > >> Ok, let's discuss about
> > public
> > > > API
> > > > > > > > design.
> > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > >>>> > >> I think we need to add
> some a
> > > > > > configure
> > > > > > > > > > entity
> > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > > > >>>> CacheConfiguration,
> > > > > > > > > > > > > > > > > > > > >>>> > >> which will contain the
> > > Compressor
> > > > > > > > interface
> > > > > > > > > > > > > > > > implementation
> > > > > > > > > > > > > > > > > > and
> > > > > > > > > > > > > > > > > > > > some
> > > > > > > > > > > > > > > > > > > > >>>> > usefull
> > > > > > > > > > > > > > > > > > > > >>>> > >> parameters.
> > > > > > > > > > > > > > > > > > > > >>>> > >> Or maybe to provide a
> > > > > > BinaryMarshaller
> > > > > > > > > > > decorator,
> > > > > > > > > > > > > > which
> > > > > > > > > > > > > > > > > will
> > > > > > > > > > > > > > > > > > be
> > > > > > > > > > > > > > > > > > > > >>>> compress
> > > > > > > > > > > > > > > > > > > > >>>> > >> data after marshalling.
> > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > >>>> > >> 2017-04-10 10:40 GMT+03:00
> > > Alexey
> > > > > > > > > Kuznetsov <
> > > > > > > > > > > > > > > > > > > > akuznetsov@apache.org
> > > > > > > > > > > > > > > > > > > > >>>> >:
> > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > >>>> > >>> Vyacheslav,
> > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > >>>> > >>> Did you read initial
> > > discussion
> > > > > [1]
> > > > > > > > about
> > > > > > > > > > > > > > compression?
> > > > > > > > > > > > > > > > > > > > >>>> > >>> As far as I remember we
> > agreed
> > > > to
> > > > > > add
> > > > > > > > only
> > > > > > > > > > > some
> > > > > > > > > > > > > > > > > "top-level"
> > > > > > > > > > > > > > > > > > > API
> > > > > > > > > > > > > > > > > > > > in
> > > > > > > > > > > > > > > > > > > > >>>> > order
> > > > > > > > > > > > > > > > > > > > >>>> > >>> to
> > > > > > > > > > > > > > > > > > > > >>>> > >>> provide a way for
> > > > > > > > > > > > > > > > > > > > >>>> > >>> Ignite users to inject
> some
> > > sort
> > > > > of
> > > > > > > > custom
> > > > > > > > > > > > > > > compression.
> > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > >>>> > >>> [1]
> > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > http://apache-ignite-developer
> > > > > > > > > > > > s.2346864.n4.nabble
> > > > > > > > > > > > > .
> > > > > > > > > > > > > > > > > > com/Data-c
> > > > > > > > > > > > > > > > > > > > >>>> > >>> ompression-in-Ignite-2-0-
> > > > > > td10099.html
> > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > >>>> > >>> On Mon, Apr 10, 2017 at
> 2:19
> > > PM,
> > > > > > > > > daradurvs <
> > > > > > > > > > > > > > > > > > > daradurvs@gmail.com
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >>>> > wrote:
> > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > Hi Igniters!
> > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > I am interested in this
> > > task.
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > Provide some kind of
> > > pluggable
> > > > > > > > > compression
> > > > > > > > > > > SPI
> > > > > > > > > > > > > > > support
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > <
> > https://issues.apache.org/
> > > > > > > > > > > > > > jira/browse/IGNITE-3592>
> > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > I developed a solution
> on
> > > > > > > > > > > > > BinaryMarshaller-level,
> > > > > > > > > > > > > > > but
> > > > > > > > > > > > > > > > > > > reviewer
> > > > > > > > > > > > > > > > > > > > >>>> has
> > > > > > > > > > > > > > > > > > > > >>>> > >>> rejected
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > it.
> > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > Let's continue
> discussion
> > of
> > > > > task
> > > > > > > > goals
> > > > > > > > > > and
> > > > > > > > > > > > > > solution
> > > > > > > > > > > > > > > > > > design.
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > As I understood that,
> the
> > > main
> > > > > > goal
> > > > > > > of
> > > > > > > > > > this
> > > > > > > > > > > > task
> > > > > > > > > > > > > > is
> > > > > > > > > > > > > > > to
> > > > > > > > > > > > > > > > > > store
> > > > > > > > > > > > > > > > > > > > >>>> data in
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > compressed form.
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > This is what I need from
> > > > Ignite
> > > > > as
> > > > > > > its
> > > > > > > > > > user.
> > > > > > > > > > > > > > > > Compression
> > > > > > > > > > > > > > > > > > > > >>>> provides
> > > > > > > > > > > > > > > > > > > > >>>> > >>> economy
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > on
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > servers.
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > We can store more data
> on
> > > same
> > > > > > > servers
> > > > > > > > > at
> > > > > > > > > > > the
> > > > > > > > > > > > > cost
> > > > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > > >>>> increasing CPU
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > utilization.
> > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > I'm researching a
> > > possibility
> > > > of
> > > > > > > > > > > > implementation
> > > > > > > > > > > > > of
> > > > > > > > > > > > > > > > > > > compression
> > > > > > > > > > > > > > > > > > > > >>>> at the
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > cache-level.
> > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > Any thoughts?
> > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > --
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > Best regards,
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > Vyacheslav
> > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > --
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > View this message in
> > > context:
> > > > > > > > > > > > > > http://apache-ignite-
> > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > developers.2346864.n4.nabble.
> > > > > > > > > > > > > > > com/Data-compression-in-
> > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > Ignite-2-0-tp10099p16317.html
> > > > > > > > > > > > > > > > > > > > >>>> > >>> > Sent from the Apache
> > Ignite
> > > > > > > Developers
> > > > > > > > > > > mailing
> > > > > > > > > > > > > > list
> > > > > > > > > > > > > > > > > > archive
> > > > > > > > > > > > > > > > > > > at
> > > > > > > > > > > > > > > > > > > > >>>> > >>> Nabble.com.
> > > > > > > > > > > > > > > > > > > > >>>> > >>> >
> > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > >>>> > >>> --
> > > > > > > > > > > > > > > > > > > > >>>> > >>> Alexey Kuznetsov
> > > > > > > > > > > > > > > > > > > > >>>> > >>>
> > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > >>>> > >> --
> > > > > > > > > > > > > > > > > > > > >>>> > >> Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > > > > >>>> > >>
> > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > >>>> > > --
> > > > > > > > > > > > > > > > > > > > >>>> > > Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > > > > >>>> > >
> > > > > > > > > > > > > > > > > > > > >>>> >
> > > > > > > > > > > > > > > > > > > > >>>> >
> > > > > > > > > > > > > > > > > > > > >>>> >
> > > > > > > > > > > > > > > > > > > > >>>> > --
> > > > > > > > > > > > > > > > > > > > >>>> > Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > > > > >>>> >
> > > > > > > > > > > > > > > > > > > > >>>>
> > > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > > >>> --
> > > > > > > > > > > > > > > > > > > > >>> Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > > > > >>>
> > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > >> --
> > > > > > > > > > > > > > > > > > > > >> Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > > > > >>
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > > Best Regards, Anton Churaev
> > > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > --
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Best Regards, Anton Churaev
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > --
> > > > > > > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > --
> > > > > > > > > > > > >
> > > > > > > > > > > > > Best Regards, Anton Churaev
> > > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > >
> > > > > > > > > > > > --
> > > > > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > --
> > > > > > > > > > >
> > > > > > > > > > > Best Regards, Anton Churaev
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > Best Regards, Vyacheslav
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Best Regards, Vyacheslav
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > >
> > > > > > Best Regards, Anton Churaev
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Best Regards, Vyacheslav
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Best Regards, Vyacheslav
> > >
> >
>
>
>
> --
> Best Regards, Vyacheslav
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message