Mailing-List: contact users-help@subversion.apache.org; run by ezmlm
Precedence: bulk
Message-ID: <57230D50.6000100@apache.org>
Date: Fri, 29 Apr 2016 09:29:20 +0200
From: Stefan Fuhrmann <stefan2@apache.org>
Organization: Apache Software Foundation
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:24.0) Gecko/20100101 Thunderbird/24.6.0
MIME-Version: 1.0
To: Johan Corveleyn <jcorvel@gmail.com>,
 Terry Dooher <Terry.Dooher@naturalmotion.com>
CC: users@subversion.apache.org
Subject: Re: 1.9.4 fix
References: <c51abb3f340e424888e8b3c11841a04e@OXEXC2.NM.local>
 <CAB84uBU-RNHHXVMAazzu7CEJGzy=wqdEnB04NorAC=GqzcMXTQ@mail.gmail.com>
In-Reply-To: 
 <CAB84uBU-RNHHXVMAazzu7CEJGzy=wqdEnB04NorAC=GqzcMXTQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit


On 28.04.2016 09:35, Johan Corveleyn wrote:
> Op 27-apr.-2016 16:17 schreef "Terry Dooher" <Terry.Dooher@naturalmotion.com
> <mailto:Terry.Dooher@naturalmotion.com>>:
>  >
>  > Hi all,
>  >
>  >
>  >
>  > I’m in the process of dump/loading our repositories to take advantage of FSFS
> format 7. Looking at the change list for 1.9.4 (out tomorrow), it shows:
>  >
>  >        fsfs: fix a rare source of incomplete dump files and reports (r1717876)
>  >
>  >
>  >
>  > I’m having trouble finding a layman’s explanation of what this issue means,
> however. (I’ve read https://issues.apache.org/jira/browse/SVN-4554 but it
> doesn’t make much sense to me.)

Hi Terry,

We recently fixed 2 issues that were caused by an idiosyncrasy
of FSFS interacting badly with recent storage saving measures.

Issue #4554 talks about two things, a set of conditions and their
immediate impact (broken dump file).  The dump file will simply
not load and that's easy enough to detect.  The set of conditions
is as follows:

   1. Representation sharing must be enabled (default since 1.6 or so).
   2. At least one change must be committed using a 1.8+ server.
   3. Property and directory deltification must not be enabled
      (available since 1.8 but enabled by default only since 1.9).
   4. A file gets committed whose contents happens to look exactly
      like a serialized hash (i.e. a directory content or property list)
      that has been committed with 1.8+.
   5. No file with the same contents has been committed using a pre-1.8
      server with rep-sharing enabled.

Point 4 is unlikely to occur by accident.  By far your best shot
at this is to have a 4-byte file containing "END\n", which happens
to match an empty hash.  Another chance is committing the contents
of a .svn folders of a pre-1.7 working copy - which requires some
serious fiddling with the .svn folders.

These 5 conditions also apply to the issue fixed by r1717876 but
the impact is different.  A change may be omitted entirely from
the dump file, if further conditions are met:

   6. The file was empty or thought to be empty (see 3. and 4.)
      before the change and empty or thought to be empty (again
      (3. and 4.) afterwards.
   7. The dump is produced by a 1.9.0 - 1.9.3 server.

E.g. if a file contains had "END\n" in r10 and changed to "" in
r11, that change would not be dumped at all.  The dump file will
not be broken but be incomplete.

  In practical terms, I’m running through several
> hundred thousand revisions across 20 repositories. Is there a chance this bug
> could happen silently and corrupt a dump in a way that wouldn’t be picked up
> during svnadmin load (or by ‘svnadmin verify’ following the load)?

'svnadmin load' will fail.  The dump has no contents for the
respective file node ('svnadmin dump' thought it was empty) but
the MD5 and SHA1 stored in the dump don't match because they refer
to the actual contents.

The best way to detect an incomplete dump is to run 'svn log -v -q'
on the repository root.  The output must be the same for the old
and new repository.

>  >
>  >
>  >
>  > I’m on 1.9.2 now. Should I wait for 1.9.4 to do this?
>  >
>
> I would use at least 1.9.3 for the process running 'svnadmin dump', because of
> this fix in 1.9.3:
>
> * svnadmin dump: preserve no-op changes (r1709388 et al, issue #4598)
>
> Otherwise your new repository might be slightly different from the original
> regarding no-op changes (see the issue [1] for more explanation).

That would be a third and unrelated issue.  It basically revolves
around the question "Does SVN need to preserve a change attempt
if it did not actually modify any contents?".  1.9.0 said "no",
1.9.3+ says "yes" again as would all older releases.

No-op changes cannot be produced by standard SVN tooling but require
direct low-level API access or some tool that writes its own dump
files (e.g. as part of the conversion from CVS to SVN).  No contents
was lost but the relation between commit message and changed paths
list got lost.

-- Stefan^2.

> [1] https://issues.apache.org/jira/plugins/servlet/mobile#issue/SVN-4598