subversion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Johan Corveleyn <jcor...@gmail.com>
Subject Re: Proposal: new fsfs.conf properties
Date Fri, 14 Jul 2017 11:38:08 GMT
On Fri, Jul 14, 2017 at 1:18 PM, Evgeny Kotkov
<evgeny.kotkov@visualsvn.com> wrote:
> Paul Hammant <paul@hammant.org> writes:
>
>> 1. compression-exempt-suffixes = mp3,mp4,jpeg
>>
>> 2. deltification-exempt-suffixes = mp3,mp4,jpeg
>>
>> Regardless of the setting of 'compression-level', #1 above two mean certain
>> things can skip the compression attempt.  It must give up at a certain point
>> right?
>
> Hi everyone,
>
> To improve the situation with slow commits of large binary and, possibly,
> incompressible files I committed a patch (http://svn.apache.org/r1801940)
> that adds initial support for LZ4 compression in the backend.  While still
> providing a decent compression ratio, LZ4 offers much faster compression
> even than zlib with level=1, and can skip incompressible data chunks.
> (Presumably, LZ4 is used for on-the-fly compression in different file
> systems for these reasons.)
>
> I have seen significant (up to 3 times) speed improvement for svn import
> and commit with large binary files.  Sometimes, using LZ4 compression can
> even outperform a configuration with disabled compression, if the file is
> at least somehow compressible, as it reduces the time required to write
> the data to disk.
>
> Currently, LZ4 compression is enabled if the fsfs.conf file specifies
> compression-level=1, and all other levels still use zlib for compression.
> Right now, the support for LZ4 is only implemented for the file:// protocol,
> but support/negotiation for other protocols can be added later.

Nice!

But how about deltification? Has anyone tried / benchmarked the effect
of turning off deltification (with or without compression), to see
what the effect would be on the commit time? Like I suggested in this
thread yesterday (i.e. set max-deltification-walk to 0 in fsfs.conf --
or perhaps play with both max-deltification-walk and
max-linear-deltification) ...

-- 
Johan

Mime
View raw message