ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ozerov <voze...@gridgain.com>
Subject Re: Transport compression (not store compression)
Date Thu, 23 Nov 2017 09:06:05 GMT
Denis,

Regarding zipped marshaller - this would be inefficient, because
compression rate will be lower.

On Thu, Nov 23, 2017 at 1:01 AM, Denis Magda <dmagda@apache.org> wrote:

> Nikita,
>
> Your solution sounds reasonable from the first glance. However, the
> communication layer processes a dozen of small system messages that should
> be excluded from the compression. Guess, that we will spend more time on
> compressing/decompressing them thus diminishing the positive effect of the
> compression.
>
> Alexey K., Vladimir O.,
>
> What if we create Zip version of the binary marshaller the same way we
> implemented GridClientZipOptimizedMarshaller?
>
> —
> Denis
>
> > On Nov 22, 2017, at 5:36 AM, Alexey Kuznetsov <akuznetsov@apache.org>
> wrote:
> >
> > I think it is very useful feature.
> > I also have experience when server nodes connected via fast network.
> > But client nodes via very slow network.
> >
> > I implemeted GridClientZipOptimizedMarshaller and that solved my issue.
> > But this marshaller works only with old
> > and org.apache.ignite.internal.client.GridClient and has a lot of
> > limitations.
> > But compression was about 6-20x times.
> >
> > We need a solution for Ignite 2.x and client nodes.
> >
> >
> > On Wed, Nov 22, 2017 at 7:48 PM, Nikita Amelchev <nsamelchev@gmail.com>
> > wrote:
> >
> >> Hello, Igniters!
> >>
> >> I think it is a useful feature. I suggest to implement it to
> communication
> >> SPI like SSL encryption implemented. I have experience with this feature
> >> and I can try to develop it.
> >>
> >> 2017-11-22 12:01 GMT+03:00 Alexey Kukushkin <kukushkinalexey@gmail.com
> >:
> >>
> >>> Forwarding to DEV list: Ignite developers, could you please share your
> >>> thoughts on how hard it is to extend Ignite to compress data on the
> >>> network.
> >>>
> >>> On Wed, Nov 22, 2017 at 10:04 AM, Gordon Reid (Nine Mile) <
> >>> gordon.reid@ninemilefinancial.com> wrote:
> >>>
> >>>> Hi Igniters,
> >>>>
> >>>>
> >>>>
> >>>> I see there is a lot of discussion in certain threads about
> >> compression.
> >>>> This seems to have diverged into conversations about object versus
> >> field
> >>>> compression, and even throwing encryption into the mix. For my use
> >> case,
> >>> I
> >>>> am not interested in compressing the cache stored in memory, I have
> >>> plenty
> >>>> of memory for my application. What I don’t have is a good network.
I
> >>> have a
> >>>> high latency, low bandwidth network between my C# ignite client and
my
> >>> Java
> >>>> ignite server. I only want to compress data when it is sent over the
> >>>> network to remote nodes. It should be stored in the local memory
> >>>> uncompressed. How can we achive this? Can the TcpCommunicationSpi
> >> support
> >>>> compression?
> >>>>
> >>>>
> >>>>
> >>>> Thanks,
> >>>>
> >>>> Gordon.
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> This email and any attachments are proprietary & confidential and
are
> >>>> intended solely for the use of the individuals to whom it is
> addressed.
> >>> Any
> >>>> views or opinions expressed are solely for those of the author and do
> >> not
> >>>> necessarily reflect those of Nine Mile Financial Pty. Limited. If you
> >>> have
> >>>> received this email in error, please let us know immediately by reply
> >>> email
> >>>> and delete from your system. Nine Mile Financial Pty. Limited. ABN:
> 346
> >>>> 1349 0252
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Best regards,
> >>> Alexey
> >>>
> >>
> >>
> >>
> >> --
> >> Best wishes,
> >> Amelchev Nikita
> >>
> >
> >
> >
> > --
> > Alexey Kuznetsov
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message