subversion-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashod Nakashian <ashodnakash...@yahoo.com>
Subject Re: Compressed Pristines (Summary)
Date Wed, 04 Apr 2012 19:26:15 GMT



>________________________________
> From: Daniel Shahaf <danielsh@elego.de>
>To: Ashod Nakashian <ashodnakashian@yahoo.com> 
>Cc: Julian Foad <julianfoad@btopenworld.com>; Markus Schaber <m.schaber@3s-software.com>;
"mtherieau@gmail.com" <mtherieau@gmail.com>; Subversion Development <dev@subversion.apache.org>

>Sent: Wednesday, April 4, 2012 10:46 PM
>Subject: Re: Compressed Pristines (Summary)
> 
>Ashod Nakashian wrote on Wed, Apr 04, 2012 at 11:41:43 -0700:
>> 
>> 
>> 
>> 
>> ----- Original Message -----
>> > From: Julian Foad <julianfoad@btopenworld.com>
>> > To: Ashod Nakashian <ashodnakashian@yahoo.com>
>> > Cc: Daniel Shahaf <danielsh@elego.de>; Markus Schaber <m.schaber@3s-software.com>;
"mtherieau@gmail.com" <mtherieau@gmail.com>; Subversion Development <dev@subversion.apache.org>
>> > Sent: Wednesday, April 4, 2012 9:45 PM
>> > Subject: Re: Compressed Pristines (Summary)
>> > 
>> > Ashod Nakashian wrote:
>> > 
>> >>  Daniel Shahaf wrote:
>> >>>>>  Would you like to assess the feasibility of compressing the

>> > pristine
>> >>>>   >store by re-mounting the "pristines" subdirectory as 
>> > a compressed
>> >>>>   >subtree in the operating system's file system?
>> >>>> 
>> >>>>   No :-)
>> >>>> 
>> >>>>   There are two ways to answer this interesting proposition of
>> >>>>   compressed file-systems. The obvious one is that it isn't 
>> > something
>> >>>>   SVN can or should control. The file-system and certainly system
>> >>>>   drivers are up to the user and any requirement or suggestion
of
>> >>>>   tempering with them is decidedly unwarranted and unexpected from
>> >>>>   a VCS.
>> >>> 
>> >>>  The suggestion wasn't that you teach svn how to change the OS's 
>> > fs
>> >>>  settinsg, it was that you investigate how solutions at the OS level
>> >>>  compare to the other approaches already suggested (custom format -based
>> >>>  and sqlite-based pristine store).
>> > 
>> > Yes, Daniel, that's exactly what I meant.  Sorry, Ash, for not being clear

>> > enough.
>> > 
>> > Of course that's inherently a non-portable solution that may not be 
>> > available on all platforms and certainly will differ.  But that's 
>> > somewhat analogous to support for password storage: in Windows land 
>> > there's a standard way to do it, in Unixy lands you have to choose your 
>> > favourite third-party subsystem and externally configure Gnome-keyring or 
>> > KDE-wallet or whatever you chose.  Of course there are other 
>> > disadvantages too, I just think it would be interesting to compare the 
>> > disadvantages with the advantages.
>> > 
>> >>  That's an easy question. The answer is that at *best* they'll do as 
>> > good 
>> >>  as in-place compression. However, in practice they'll do much worse.
>> > 
>> > Ash, what's your measure of "best" here -- just the compression 
>> > ratio?  What about other kinds of goodness such as the automatic recovering
of 
>> > free space, and the ability to present a plain-text view of the file content

>> > through the standard filesystem API which eliminates the need for us to 
>> > implement a plain-text cache in Subversion?
>> > 
>> >>  The 
>> >>  reason is that the OS level compression works on not only the single file

>> > level, 
>> >>  but actually at the block level. This is to make modifications reasonably

>> > fast 
>> >>  (read compressed data, uncompress, modify, write recompressed data). If
the 
>> > 
>> >>  complete file is compressed then even changing a single byte (neglecting

>> > that no 
>> >>  storage works on the byte-level anyway) will yield performance that will
at 
>> > 
>> >>  least linearly degrade by the filesize.
>> > 
>> > Changing a single byte is an irrelevant scenario, because pristine files are

>> > always created in full and then never modified.
>> 
>> But the OS compression doesn't know that! It's design assumes average
>> file modification operations, and that's rarely replacing the full
>> file (which is the norm for us). I was only explaining and justifying
>> the design decision of OS level compression.
>> 
>
>But one _could_ write a filesystem that implements compression and/or
>deduplication aimed at .svn/pristine/'s workflow.  Should effort be
>spent there rather than on implementing a smarter svn_wc__pristine_*()?

Perhaps we should, but is that a simpler/easier/more-maintainable/first-try approach that
we can defend with a straight face? I'm having a hard time doing that in my current mode.

>
>Daniel
>(devil's advocate mode)
>
>> > 
>> >>>  In short: if 'mount -o compress=yes' provides 90% space savings 
>> > then 
>> >>  we
>> >>>  would have little reason to implement space-saving solutions in svn

>> > itself.
>> >>>  But it's the user's, not svn's, responsibility to run that.
>> > 
>> > By the way, Daniel, I'm not ruling out the possibility that we may want to 
>> > provide some glue logic so that Subversion can help the user to set up the 
>> > feature.  Without such assistance, only expert users would ever benefit.
>> > 
>> > It's clearly an interesting topic!
>> > 
>> > - Julian
>> >
>
>
>

Mime
View raw message