subversion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Evgeny Kotkov <evgeny.kot...@visualsvn.com>
Subject Re: Proposal: new fsfs.conf properties
Date Fri, 14 Jul 2017 11:18:27 GMT
Paul Hammant <paul@hammant.org> writes:

> 1. compression-exempt-suffixes = mp3,mp4,jpeg
>
> 2. deltification-exempt-suffixes = mp3,mp4,jpeg
>
> Regardless of the setting of 'compression-level', #1 above two mean certain
> things can skip the compression attempt.  It must give up at a certain point
> right?

Hi everyone,

To improve the situation with slow commits of large binary and, possibly,
incompressible files I committed a patch (http://svn.apache.org/r1801940)
that adds initial support for LZ4 compression in the backend.  While still
providing a decent compression ratio, LZ4 offers much faster compression
even than zlib with level=1, and can skip incompressible data chunks.
(Presumably, LZ4 is used for on-the-fly compression in different file
systems for these reasons.)

I have seen significant (up to 3 times) speed improvement for svn import
and commit with large binary files.  Sometimes, using LZ4 compression can
even outperform a configuration with disabled compression, if the file is
at least somehow compressible, as it reduces the time required to write
the data to disk.

Currently, LZ4 compression is enabled if the fsfs.conf file specifies
compression-level=1, and all other levels still use zlib for compression.
Right now, the support for LZ4 is only implemented for the file:// protocol,
but support/negotiation for other protocols can be added later.


Regards,
Evgeny Kotkov

Mime
View raw message