Mailing-List: contact dev-help@subversion.apache.org; run by ezmlm
Precedence: bulk
From: Julian Foad <julian@assembla.com>
Subject: Shelving: beyond text changes
To: dev@subversion.apache.org, svn-core@assembla.com
Message-ID: <8d858274-7d5a-9ae6-aaaa-50a6af9bd14f@assembla.com>
Date: Wed, 1 Nov 2017 10:58:58 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.4.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-GB
Content-Transfer-Encoding: 7bit
archived-at: Wed, 01 Nov 2017 10:59:09 -0000

Shelving v1 (as in what I have done so far) supports text changes only.

== What next? ==

Several possible directions:

   * Missing Support in Shelving (binary files, copies/moves, dirs)
   * Checkpointing
   * Integration with Changelists
   * Supporting Cloud-based Code Review Systems
   * Alternative Storage in WC metadata

=== Missing Support in Shelving ===

To move Shelving from a "starting point" to "fair usability" we will 
need a v2 that supports some or all of:

   binary files (and binary property values)
   copies and moves
   directory changes (mkdir, rmdir, properties)
   mergeinfo changes

See the "Straight Ahead" section below for details.

=== Checkpointing ===

Initial conversations on Checkpointing were headed in the direction of 
what we might call a local branch, with full support for the versioning 
operations that Subversion supports on a server branch. However this 
would be a huge development.

The latest thinking on Checkpointing was that the quickest way to start 
is a thin wrapper around shelving, emulating the manual approach of 
saving a series of better and better patches:

   'foo-1.patch' 'foo-2.patch' etc.

Commands:

   'svn checkpoint' increments the suffix number on the last used 
(shelved) name and runs 'svn shelve --keep-local <NAME>-<NUMBER>'.
   'svn checkpoint revert <NUMBER>' runs 'svn revert; svn unshelve 
<NAME>-<NUMBER>'.

No support for commit-all-as-separate-commits, nor rebasing after 'svn 
update', nor rolling back the base state to undo an update.

An initial version of this could be done in as little as ~2 weeks.

=== Integration with Changelists ===

The idea of a Change-Set is currently split 3 ways in Subversion: 
commits, changelists, and now shelving. We should make this more 
comprehensible by bringing these strands together, as discussed in the 
section in Shelving-Checkpointing Dev:


https://docs.google.com/document/d/1PVgw0BdPF7v67oxIK7B_Yjmr3p28ojabP5N1PfZTsHk/edit#heading=h.ay5h5pfumpv8

=== Supporting Cloud-based Code Review Systems ===

Paul Hammant has noted how years ago Google got started building a 
better code review system by scripting to send local changes from a 
Perforce WC as patches to a review server. The Subversion SAS companies 
like Assembla and RhodeCode now have code review and merge request 
systems for Subversion. How can we improve support in this direction, 
perhaps building on Shelving? Perhaps defining standards for such metadata?

=== Alternative Storage ===

An alternative is to store shelved changes in the same way as the WC 
metadata already stores the WC working changes. Advantages:

   * speed & space efficiency
   * include the WC base version to enable 3-way merge.

The "missing support" in shelving (binary files, copies and moves, 
directory changes, mergeinfo changes) will come along "for free" because 
the WC already supports them.

Instead, the work is in extending the WC infrastructure. I believe the 
effort needed to complete this is large and uncertain. A smaller effort 
in this direction could yield benefits in the way of making the pristine 
store more flexible, towards:

   optional or compressed pristines (a separate wish-list feature)
   using the pristine store also for the base of shelved changes
   using the pristine store for shelved modified files,
     -> much faster for large binary files


== The "Straight ahead" implementation of Missing Support in Shelving ==

The "Straight ahead" route is to Enhance "svn diff" and "svn patch" 
feature-by-feature until they support all the required features. Here is 
more detail on the steps required.

   binary files (moderate effort, say ~4 weeks)

     switch to 'git diff' mode
     improve read/write speed of binary patches (for speed)
     implement git-binary-diff delta mode (for space)

   copies and moves (larger effort, more uncertainty)

     TBD...

   mkdir/rmdir/dir-props (moderate effort, say ~4 weeks)

     invent a syntax (probably like that for a symlink or empty file)
     implement in 'diff'
     implement in 'patch'
     implement on/off control (e.g. diff --with-dir-changes)

   mergeinfo changes (smaller effort, say ~1 weeks)

     implement in 'patch': connect existing mergeinfo parser to make 
prop changes

Note: patch files are still inherently a sub-optimal design for large 
binary files: not as fast as is possible with other designs.


- Julian