subversion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Shahaf <danie...@elego.de>
Subject Re: Compressed Pristines (Summary)
Date Wed, 04 Apr 2012 18:46:17 GMT
Ashod Nakashian wrote on Wed, Apr 04, 2012 at 11:41:43 -0700:
> 
> 
> 
> 
> ----- Original Message -----
> > From: Julian Foad <julianfoad@btopenworld.com>
> > To: Ashod Nakashian <ashodnakashian@yahoo.com>
> > Cc: Daniel Shahaf <danielsh@elego.de>; Markus Schaber <m.schaber@3s-software.com>;
"mtherieau@gmail.com" <mtherieau@gmail.com>; Subversion Development <dev@subversion.apache.org>
> > Sent: Wednesday, April 4, 2012 9:45 PM
> > Subject: Re: Compressed Pristines (Summary)
> > 
> > Ashod Nakashian wrote:
> > 
> >>  Daniel Shahaf wrote:
> >>>>>  Would you like to assess the feasibility of compressing the 
> > pristine
> >>>>   >store by re-mounting the "pristines" subdirectory as 
> > a compressed
> >>>>   >subtree in the operating system's file system?
> >>>> 
> >>>>   No :-)
> >>>> 
> >>>>   There are two ways to answer this interesting proposition of
> >>>>   compressed file-systems. The obvious one is that it isn't 
> > something
> >>>>   SVN can or should control. The file-system and certainly system
> >>>>   drivers are up to the user and any requirement or suggestion of
> >>>>   tempering with them is decidedly unwarranted and unexpected from
> >>>>   a VCS.
> >>> 
> >>>  The suggestion wasn't that you teach svn how to change the OS's 
> > fs
> >>>  settinsg, it was that you investigate how solutions at the OS level
> >>>  compare to the other approaches already suggested (custom format -based
> >>>  and sqlite-based pristine store).
> > 
> > Yes, Daniel, that's exactly what I meant.  Sorry, Ash, for not being clear 
> > enough.
> > 
> > Of course that's inherently a non-portable solution that may not be 
> > available on all platforms and certainly will differ.  But that's 
> > somewhat analogous to support for password storage: in Windows land 
> > there's a standard way to do it, in Unixy lands you have to choose your 
> > favourite third-party subsystem and externally configure Gnome-keyring or 
> > KDE-wallet or whatever you chose.  Of course there are other 
> > disadvantages too, I just think it would be interesting to compare the 
> > disadvantages with the advantages.
> > 
> >>  That's an easy question. The answer is that at *best* they'll do as 
> > good 
> >>  as in-place compression. However, in practice they'll do much worse.
> > 
> > Ash, what's your measure of "best" here -- just the compression 
> > ratio?  What about other kinds of goodness such as the automatic recovering of

> > free space, and the ability to present a plain-text view of the file content 
> > through the standard filesystem API which eliminates the need for us to 
> > implement a plain-text cache in Subversion?
> > 
> >>  The 
> >>  reason is that the OS level compression works on not only the single file 
> > level, 
> >>  but actually at the block level. This is to make modifications reasonably 
> > fast 
> >>  (read compressed data, uncompress, modify, write recompressed data). If the

> > 
> >>  complete file is compressed then even changing a single byte (neglecting 
> > that no 
> >>  storage works on the byte-level anyway) will yield performance that will at

> > 
> >>  least linearly degrade by the filesize.
> > 
> > Changing a single byte is an irrelevant scenario, because pristine files are 
> > always created in full and then never modified.
> 
> But the OS compression doesn't know that! It's design assumes average
> file modification operations, and that's rarely replacing the full
> file (which is the norm for us). I was only explaining and justifying
> the design decision of OS level compression.
> 

But one _could_ write a filesystem that implements compression and/or
deduplication aimed at .svn/pristine/'s workflow.  Should effort be
spent there rather than on implementing a smarter svn_wc__pristine_*()?

Daniel
(devil's advocate mode)

> > 
> >>>  In short: if 'mount -o compress=yes' provides 90% space savings 
> > then 
> >>  we
> >>>  would have little reason to implement space-saving solutions in svn 
> > itself.
> >>>  But it's the user's, not svn's, responsibility to run that.
> > 
> > By the way, Daniel, I'm not ruling out the possibility that we may want to 
> > provide some glue logic so that Subversion can help the user to set up the 
> > feature.  Without such assistance, only expert users would ever benefit.
> > 
> > It's clearly an interesting topic!
> > 
> > - Julian
> >

Mime
View raw message