Return-Path: 
 <solr-dev-return-13041-apmail-lucene-solr-dev-archive=lucene.apache.org@lucene.apache.org>
Delivered-To: apmail-lucene-solr-dev-archive@locus.apache.org
Received: (qmail 9944 invoked from network); 17 Nov 2008 04:41:05 -0000
Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2)
  by minotaur.apache.org with SMTP; 17 Nov 2008 04:41:05 -0000
Received: (qmail 62505 invoked by uid 500); 17 Nov 2008 04:41:12 -0000
Delivered-To: apmail-lucene-solr-dev-archive@lucene.apache.org
Received: (qmail 62465 invoked by uid 500); 17 Nov 2008 04:41:12 -0000
Mailing-List: contact solr-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:solr-dev-help@lucene.apache.org>
List-Unsubscribe: <mailto:solr-dev-unsubscribe@lucene.apache.org>
List-Post: <mailto:solr-dev@lucene.apache.org>
List-Id: <solr-dev.lucene.apache.org>
Reply-To: solr-dev@lucene.apache.org
Delivered-To: mailing list solr-dev@lucene.apache.org
Received: (qmail 62453 invoked by uid 99); 17 Nov 2008 04:41:12 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Sun, 16 Nov 2008 20:41:12 -0800
X-ASF-Spam-Status: No, hits=-1998.8 required=10.0
	tests=ALL_TRUSTED,FS_REPLICA
X-Spam-Check-By: apache.org
Received: from [140.211.11.140] (HELO brutus.apache.org) (140.211.11.140)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 17 Nov 2008 04:39:59 +0000
Received: from brutus (localhost [127.0.0.1])
	by brutus.apache.org (Postfix) with ESMTP id 4B2D3234C282
	for <solr-dev@lucene.apache.org>; Sun, 16 Nov 2008 20:40:44 -0800 (PST)
Message-ID: <2106276394.1226896844306.JavaMail.jira@brutus>
Date: Sun, 16 Nov 2008 20:40:44 -0800 (PST)
From: "Noble Paul (JIRA)" <jira@apache.org>
To: solr-dev@lucene.apache.org
Subject: [jira] Updated: (SOLR-829) replication Compression
In-Reply-To: <639777829.1225292264270.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-Virus-Checked: Checked by ClamAV on apache.org


     [ https://issues.apache.org/jira/browse/SOLR-829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Noble Paul updated SOLR-829:
----------------------------

    Attachment: solr-829.patch

The server(Replicationhandler is agnostic of compression. The client (SnapPuller) sets appropriate header  before sending the request . Use appropriate filter or front-end the master w/ an apache to handle compression

> replication Compression
> -----------------------
>
>                 Key: SOLR-829
>                 URL: https://issues.apache.org/jira/browse/SOLR-829
>             Project: Solr
>          Issue Type: Improvement
>          Components: replication (java)
>            Reporter: Simon Collins
>            Assignee: Shalin Shekhar Mangar
>         Attachments: email discussion.txt, solr-829.patch, solr-829.patch
>
>
> From a discussion on the mailing list solr-user, it would be useful to have an option to compress the files sent between servers for replication purposes.
> Files sent across between indexes can be compressed by a large margin allowing for easier replication between sites.
> ...Noted by Noble Paul 
> we will use a gzip on both ends of the pipe . On the slave side you can say <str name="zip">true<str> as an extra option to compress and send data from server 
> Other thoughts on issue: 
> Do keep in mind that compression is a CPU intensive process so it is a trade off between CPU utilization and network bandwidth.  I have see cases where compressing the data before a network transfer ended up being slower than without compression because the cost of compression and un-compression was more than the gain in network transfer.
> Why invent something when compression is standard in HTTP? --wunder

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.