Mailing-List: contact users-help@subversion.apache.org; run by ezmlm
Precedence: bulk
Subject: Re: Cannot add files due to hierarchical RDC svn:auto-props when
 matching rules merge properties between text and binary file types
To: Rob Hofer <rob.hofer@rockwellcollins.com>, users@subversion.apache.org
References: <CAHtGZg_wsneC++O=EFCc9GUpm8xKivJCE848oeOpw5T_59ke9A@mail.gmail.com>
Cc: Rob Hofer <rob@hofers.us>
From: Eric Johnson <eric@tibco.com>
Message-ID: <66adfc9d-8929-c5b5-c871-ba5d088ba1bd@tibco.com>
Date: Wed, 12 Oct 2016 17:15:48 -0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:45.0)
 Gecko/20100101 Thunderbird/45.4.0
MIME-Version: 1.0
In-Reply-To: <CAHtGZg_wsneC++O=EFCc9GUpm8xKivJCE848oeOpw5T_59ke9A@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
archived-at: Thu, 13 Oct 2016 00:15:58 -0000

Your constraints, as currently specified, seem to require actual logic....

Thoughts follow your email.

On 10/12/16 1:44 PM, Rob Hofer wrote:
> We have a rather common use case where we have an svn:auto-props rule 
> set globally (set on root of repository) to define source code files 
> as text based, but also have some files provided by 3rd parties which 
> compress or encrypt similar files with the same file extension (which 
> we have no control over), and hence we want to set an svn:auto-props 
> rule locally on those directories to make those files binary type. But 
> the hierarchical svn:auto-props rules add properties from multiple 
> definitions of the same match filter (*.v), and result is a 
> conflicting set of properties that block the add at the client 
> (eol-style with mime-type=octet-stream).
>
> For example, a binary and text based Verilog file (*.v):
> %> file binary.v
> binary.v: gzip compressed data, was "binary.v", from Unix, last 
> modified: Mon Feb 18 19:44:25 2013, max compression
> %> file text.v
> text.v: ASCII text
>
> The RDC auto-props for this directory shows these Verilog and VHDL 
> types (local directory expects binary types, global are source-code 
> text files).
> %> svn propget svn:auto-props --show-inherited-props .
> http://crsvn/gsadc - *.v      = svn:eol-style=LF;svn:keywords=Id 
> Revision Date Author URL;svn:mime-type=text/x-verilog
> . - *.v   = svn:needs-lock;svn:mime-type=application/octet-stream
>
> Adding the files to SVN control fails, unless --no-auto-props is used 
> (undesirable work around):
> %> svn add binary.v
> svn: E200009: Can't set 'svn:eol-style': file '/module/lay/binary.v' 
> has binary mime type property
> %> svn add --no-auto-props binary.v
> A  (bin)  binary.v
> %> svn add text.v
> svn: E200009: Can't set 'svn:eol-style': file '/module/lay/text.v' has 
> binary mime type property
> %> svn add --no-auto-props text.v
> A         text.v
>
> Subversion auto-detects the binary file and at least sets the 
> mime-type, but other properties are missing because --no-auto-props is 
> the only way to add the files without error.
> %> svn proplist -v binary.v
> Properties on 'binary.v':
>   svn:mime-type
>     application/octet-stream
> %> svn proplist -v text.v
>
> %> svn --version
> svn, version 1.9.4 (r1740329)
>    compiled May 18 2016, 12:05:49 on x86_64-unknown-linux-gnu
>
> (Incidentally, the commit will fail because our hook is checking these 
> svn:auto-props rules and the file must match the binary rules or the 
> text rule, based on the mime-type). So the only way today to add these 
> files to SVN is to use --no-auto-props on add, and go into the server 
> and disable the pre-commit hook during the commit, then put the 
> pre-commit hook back.  Since a pre-commit hook is the only way to 
> enforce the use of auto-props correctly, disabling the hook is not an 
> optimal solution. Once added to SVN, the issue never comes up again 
> because the properties do not change, and the pre-commit hook checks 
> the modifications on future commits based upon the mime-type (binary 
> or text rules). The issue is only during the initial add to SVN due to 
> conflicting properties being applied to the file based on how the 
> svn:auto-props definitions are being interpreted.
>
> Proposed solution:
> 1. Make lower level svn:auto-props rules completely override upstream 
> ones, rather than additively merging properties, for rules with same 
> exact match filter (local *.v redefines upstream *.v completely).
> 2. Make SVN ignore properties such as eol-style and keywords when the 
> mime-type is a binary type rather than issue a fatal error to the 
> user/client (warning instead that some properties are being ignored).
> 3. Provide a subtractive property rule to undo an upstream property. 
> E.g., svn:eol-style=none, or svn:keyword=none, such that a lower level 
> rule can unset a property defined upstream (and make 
> svn:eol-style=none behave same as if svn:eol-style was not applied at 
> all).
>
> Note: Before RDC svn:auto-props (pre 1.8), this use case required 
> having two entries in the ~/.subversion/config, with one always 
> commented out. When encountering one type or the other (text or 
> binary), user would have to uncomment/comment the proper auto-props 
> rule in their config file before the add, and then change their config 
> back for the normal case. This was very unwieldly and required careful 
> synchronization of all user runtime config files and hook scripts and 
> manual intervention by the user during adds.  Hierarchical RDC 
> properties should eliminate this problem, but it falls a little short 
> because of how hierarchical RDC svn:auto-props rules merge mutually 
> exclusive properties together. I believe this is very similar to 
> multiple matches in the ~/.subversion/config runtime file, for example 
> a *.v rule and a *-rtl.v rule could collide, except now it is possible 
> to have the very same rule filter (*.v) defined in more than one 
> location in the subversion hierarchy. See proposed solution #1 as my 
> desired behavior from the SVN client.


Only a few options I see:

  - different repositories for the binary files vs. the text format.

  - improved hook scripts to enforce your desired constraints based on 
the content of the incoming added file, rather than just the extension. 
You could also create a client side script / tool to check before a commit.

Eric.