Mailing-List: contact dev-help@subversion.apache.org; run by ezmlm
Precedence: bulk
MIME-Version: 1.0
In-Reply-To: <20170711200003.nnjvk6zimkpb4y6j@tarpaulin.shahaf.local2>
References: <CA+298Uhy6TvBJ9hF=BCvViiyAQbgkj4Zwe-0DfCM3=zZL-r_6g@mail.gmail.com>
 <CAEyhmwYka2O6BaBbtmce-ELuPek5t3mcwjc0JsMfb7z5eR5+kw@mail.gmail.com>
 <727D8E16AE957149B447FE368139F2B5A9023507@SERVER10> <CA+298UhPVgFn9rAaj8zoh54_m3G4irVwPjXiGmHtVm0KPcaDaw@mail.gmail.com>
 <727D8E16AE957149B447FE368139F2B5A9023540@SERVER10> <20170711200003.nnjvk6zimkpb4y6j@tarpaulin.shahaf.local2>
From: Paul Hammant <paul@hammant.org>
Date: Tue, 11 Jul 2017 16:07:24 -0400
Message-ID: <CA+298UjB4RPOe9RMoN1TJTyvOY43x4KCkFPMUfe7gWRvN13tTA@mail.gmail.com>
Subject: Re: Proposal: new fsfs.conf properties
To: Daniel Shahaf <danielsh@apache.org>
Cc: Markus Schaber <m.schaber@codesys.com>, Pavel Lyalyakin <pavel.lyalyakin@visualsvn.com>,
	Subversion Development <dev@subversion.apache.org>
Content-Type: multipart/alternative; boundary="001a11c171e67e10090554103f3b"
archived-at: Tue, 11 Jul 2017 20:07:30 -0000

--001a11c171e67e10090554103f3b
Content-Type: text/plain; charset="UTF-8"

So I'm after a time saving. I'm perfectly happy for the backend to waste
space (in my configuration), I just don't want it to take 15 mins to
transfer a single 15GB file into Subversion.

In my configuration, I'd like to pre-advise Subversion to save as much time
as possible for uploads, by skipping steps that are known in advance to be
meaningless for the use case.

- Paul

On Tue, Jul 11, 2017 at 4:00 PM, Daniel Shahaf <danielsh@apache.org> wrote:

> On Tue, Jul 11, 2017 at 01:39:56PM +0000, Markus Schaber wrote:
> > To summarize it up:
> >
> > I expect significant benefits in some use cases by skipping the
> > compression, thus I'm +1 if benchmarks prove it's worth the effort.
>
> It is easy to have deltification without compression, either by using
> svndiff0 (instead of svndiff1) or by using svndiff1 with zlib
> compression level set appropriately.
>
> > I see the danger of drastically increased bandwith and storage size
> > (transferring/storing the whole mp3 instead of just some changed meta
> > data bytes) in some common use cases when deltification is skipped.
> > Thus, I'm skeptical (count it as -0), and I'd kindly suggest to do
> > some benchmarks for those cases before implementation, and clear
> > documentation of the possible negative effects if it's implemented.
>
> Regarding compression, would it make sense for the server to compute the
> compressed delta, and it turns out to be larger than X% of the
> uncompressed&undeltified file, to just store the latter?  I.e., to
> compute the DELTA rep but use a PLAIN rep if the DELTA rep would be
> larger than X% (in bytes) of the PLAIN rep?
>
> IIRC this is already so with X=100, but for some filetypes it might make
> sense to set X lower.
>
> Cheers,
>
> Daniel
>

--001a11c171e67e10090554103f3b
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">So I&#39;m after a time saving. I&#39;m perfectly happy fo=
r the backend to waste space (in my configuration), I just don&#39;t want i=
t to take 15 mins to transfer a single 15GB file into Subversion.<div><br><=
/div><div>In my configuration, I&#39;d like to pre-advise Subversion to sav=
e as much time as possible for uploads, by skipping steps that are known in=
 advance to be meaningless for the use case.</div><div><span style=3D"font-=
size:12.8px"><br></span></div><div><span style=3D"font-size:12.8px">- Paul=
=C2=A0</span><br></div></div><div class=3D"gmail_extra"><br><div class=3D"g=
mail_quote">On Tue, Jul 11, 2017 at 4:00 PM, Daniel Shahaf <span dir=3D"ltr=
">&lt;<a href=3D"mailto:danielsh@apache.org" target=3D"_blank">danielsh@apa=
che.org</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D=
"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><span class=
=3D"">On Tue, Jul 11, 2017 at 01:39:56PM +0000, Markus Schaber wrote:<br>
&gt; To summarize it up:<br>
&gt;<br>
&gt; I expect significant benefits in some use cases by skipping the<br>
&gt; compression, thus I&#39;m +1 if benchmarks prove it&#39;s worth the ef=
fort.<br>
<br>
</span>It is easy to have deltification without compression, either by usin=
g<br>
svndiff0 (instead of svndiff1) or by using svndiff1 with zlib<br>
compression level set appropriately.<br>
<span class=3D""><br>
&gt; I see the danger of drastically increased bandwith and storage size<br=
>
&gt; (transferring/storing the whole mp3 instead of just some changed meta<=
br>
&gt; data bytes) in some common use cases when deltification is skipped.<br=
>
&gt; Thus, I&#39;m skeptical (count it as -0), and I&#39;d kindly suggest t=
o do<br>
&gt; some benchmarks for those cases before implementation, and clear<br>
&gt; documentation of the possible negative effects if it&#39;s implemented=
.<br>
<br>
</span>Regarding compression, would it make sense for the server to compute=
 the<br>
compressed delta, and it turns out to be larger than X% of the<br>
uncompressed&amp;undeltified file, to just store the latter?=C2=A0 I.e., to=
<br>
compute the DELTA rep but use a PLAIN rep if the DELTA rep would be<br>
larger than X% (in bytes) of the PLAIN rep?<br>
<br>
IIRC this is already so with X=3D100, but for some filetypes it might make<=
br>
sense to set X lower.<br>
<br>
Cheers,<br>
<br>
Daniel<br>
</blockquote></div><br></div>

--001a11c171e67e10090554103f3b--