lucene-solr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Noble Paul (JIRA)" <j...@apache.org>
Subject [jira] Updated: (SOLR-829) replication Compression
Date Mon, 17 Nov 2008 04:40:44 GMT

     [ https://issues.apache.org/jira/browse/SOLR-829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Noble Paul updated SOLR-829:
----------------------------

    Attachment: solr-829.patch

The server(Replicationhandler is agnostic of compression. The client (SnapPuller) sets appropriate
header  before sending the request . Use appropriate filter or front-end the master w/ an
apache to handle compression

> replication Compression
> -----------------------
>
>                 Key: SOLR-829
>                 URL: https://issues.apache.org/jira/browse/SOLR-829
>             Project: Solr
>          Issue Type: Improvement
>          Components: replication (java)
>            Reporter: Simon Collins
>            Assignee: Shalin Shekhar Mangar
>         Attachments: email discussion.txt, solr-829.patch, solr-829.patch
>
>
> From a discussion on the mailing list solr-user, it would be useful to have an option
to compress the files sent between servers for replication purposes.
> Files sent across between indexes can be compressed by a large margin allowing for easier
replication between sites.
> ...Noted by Noble Paul 
> we will use a gzip on both ends of the pipe . On the slave side you can say <str name="zip">true<str>
as an extra option to compress and send data from server 
> Other thoughts on issue: 
> Do keep in mind that compression is a CPU intensive process so it is a trade off between
CPU utilization and network bandwidth.  I have see cases where compressing the data before
a network transfer ended up being slower than without compression because the cost of compression
and un-compression was more than the gain in network transfer.
> Why invent something when compression is standard in HTTP? --wunder

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message