giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pavan Kumar (JIRA)" <>
Subject [jira] [Updated] (GIRAPH-713) Provide an option to do request compression
Date Fri, 06 Jun 2014 19:52:01 GMT


Pavan Kumar updated GIRAPH-713:

    Attachment: GIRAPH-713_1.patch

currently netty has native support for jdkzlib & snappy
I think jdkzlib might be better because if it is on hotpatch then jdk can replace it with
native methods, since it is uses for deflater & inflater. On the other hand lz4
is a really fast algorithm for compression-decompression, but not sure how it pans out against
a native (which jdk zip can potentially use instead of just making changes at the java application

I am attaching a patch that I used to do a few test runs on the cluster.
It works for snappy, as well as for no compression, but fails with an exception for jdkzlib
If some one would like to work on it, please feel free to start off with the patch. Thanks.

> Provide an option to do request compression
> -------------------------------------------
>                 Key: GIRAPH-713
>                 URL:
>             Project: Giraph
>          Issue Type: New Feature
>            Reporter: Maja Kabiljo
>         Attachments: GIRAPH-713.patch, GIRAPH-713_1.patch
> In some cases, network is much slower than all the computation stuff we do, and we could
benefit from compressing requests.

This message was sent by Atlassian JIRA

View raw message