giraph-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pavan Kumar (JIRA)" <>
Subject [jira] [Commented] (GIRAPH-713) Provide an option to do request compression
Date Thu, 03 Jul 2014 23:45:34 GMT


Pavan Kumar commented on GIRAPH-713:

I created a patch for netty upgrade on the dependent task
(note that the additional changes are because we do not want to use cache - some internal
testing showed high perf regression with caching turned on in netty)

ratio of sent bytes with different compression algos for pagerank run on a graph with 1million
vertices and 85 million edges

none vs jdkzlib = 2.5
none vs snappy = 1.5

however in my runs, the overhead of compression is much  more than saving obtained. this is
not a comprehensive experiment, though.
our network is much faster, etc. so might be useful for others, in anycase, the change is
non-intrusive, by default compression is turned off, so no regression. Users, can turn on
compression by providing the option 

> Provide an option to do request compression
> -------------------------------------------
>                 Key: GIRAPH-713
>                 URL:
>             Project: Giraph
>          Issue Type: New Feature
>            Reporter: Maja Kabiljo
>         Attachments: GIRAPH-713.patch, GIRAPH-713_1.patch
> In some cases, network is much slower than all the computation stuff we do, and we could
benefit from compressing requests.

This message was sent by Atlassian JIRA

View raw message