subversion-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Attila Nagy <...@fsn.hu>
Subject Re: Assertion failed and crash with 1.7.1
Date Thu, 27 Oct 2011 12:10:48 GMT
On 10/27/11 13:47, Mark Phippard wrote:
> On Thu, Oct 27, 2011 at 4:47 AM, Attila Nagy <bra@fsn.hu 
> <mailto:bra@fsn.hu>> wrote:
>
>     On 10/26/11 15:37, Philip Martin wrote:
>>     Attila Nagy<bra@fsn.hu>  <mailto:bra@fsn.hu>  writes:
>>
>>>     I'm trying to update a working copy of some tens of GBs with svn 1.7.1
>>     Did you upgrade with 1.7.0 or 1.7.1?
>     I've upgraded the WC with 1.7.0, then switched to 1.7.1, which I'm
>     currently using.
>     The upgrade took nearly one week (I can't afford a clean checkout,
>     because I have to preserve the files' inode numbers), at the start
>     it was running very fast, and at the end of the week it could
>     print a new directory in every 8-10 seconds, while the svn
>     processes CPU usage was 100%.
>     The disks weren't touched, it seems that even with a database
>     completely in the OS's buffer cache the SQL operations are pretty
>     slow with a lot of files.
>     Could it be because some indexes are missing (or not optimized for
>     this amount of files), or it is what we could get from SQLite?
>
>
> I can see where having all that RAM could help greatly when the 
> database is being read, but I do not see how it would help when we are 
> writing to the database.  I would imagine the database does stuff to 
> flush itself to disk whenever a transaction is committed.  I thought 
> this was why we see 1.7 slower than other versions via NFS, even on 
> small working copies where the database would fit in anyones RAM.
>
> I would imagine svn upgrade is almost entirely writes and I recall it 
> does quite a few transactions.  So couldn't the linear slow down just 
> be based on the growth in the amount of bytes that are written to disk 
> each time?
>
I've checked the disk IO stat and I couldn't see any bottlenecks there. 
In fact while upgrade-ing, I barely saw any disk IO.
Checking the process revealed a lot of random reads from the wc.db, 
which the OS could serve out of memory.
BTW, I don't have any transaction-related files next to my wc.db (I 
remember about wal, or journal named files, but I'm not a regular user 
of SQLite), and definitely couldn't see too much fsyncs.

Mime
View raw message