subversion-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From danie...@apache.org
Subject svn commit: r1163441 - in /subversion/branches/revprop-packing: BRANCH-README subversion/libsvn_fs_fs/fs.h subversion/libsvn_fs_fs/fs_fs.c subversion/libsvn_fs_fs/structure
Date Tue, 30 Aug 2011 22:43:43 GMT
Author: danielsh
Date: Tue Aug 30 22:43:43 2011
New Revision: 1163441

URL: http://svn.apache.org/viewvc?rev=1163441&view=rev
Log:
On the revprop-packing branch, implement a revprop pack file format number.

* subversion/libsvn_fs_fs/structure
  (Packing revision properties): Document the pack file formats.

* subversion/libsvn_fs_fs/fs.h
  (SVN_FS_FS__REVPROP_FORMAT_NUMBER): New macro.

* subversion/libsvn_fs_fs/fs_fs.c
  (check_revprop_format, copy_first_line): New helpers.
  (set_revision_proplist): Document assumptions. Copy the format number.
  (revision_proplist): Read nad check the format number.
  (pack_revprop_shard): Write a format number.

* BRANCH-README: Remove this task.

Modified:
    subversion/branches/revprop-packing/BRANCH-README
    subversion/branches/revprop-packing/subversion/libsvn_fs_fs/fs.h
    subversion/branches/revprop-packing/subversion/libsvn_fs_fs/fs_fs.c
    subversion/branches/revprop-packing/subversion/libsvn_fs_fs/structure

Modified: subversion/branches/revprop-packing/BRANCH-README
URL: http://svn.apache.org/viewvc/subversion/branches/revprop-packing/BRANCH-README?rev=1163441&r1=1163440&r2=1163441&view=diff
==============================================================================
--- subversion/branches/revprop-packing/BRANCH-README (original)
+++ subversion/branches/revprop-packing/BRANCH-README Tue Aug 30 22:43:43 2011
@@ -27,8 +27,6 @@ TODO:
 * add some explicit tests for svnadmin upgrade/pack
   (both revs and revrops)
 
-* format number at the top of the pack file?
-
 * cf fs-successor-ids plans:
     Date: Fri, 26 Aug 2011 13:14:32 +0200
     From: Stefan Sperling <stsp@elego.de>

Modified: subversion/branches/revprop-packing/subversion/libsvn_fs_fs/fs.h
URL: http://svn.apache.org/viewvc/subversion/branches/revprop-packing/subversion/libsvn_fs_fs/fs.h?rev=1163441&r1=1163440&r2=1163441&view=diff
==============================================================================
--- subversion/branches/revprop-packing/subversion/libsvn_fs_fs/fs.h (original)
+++ subversion/branches/revprop-packing/subversion/libsvn_fs_fs/fs.h Tue Aug 30 22:43:43 2011
@@ -132,6 +132,10 @@ extern "C" {
 /* The minimum format number that supports a configuration file (fsfs.conf) */
 #define SVN_FS_FS__MIN_CONFIG_FILE 4
 
+/* The format number of revprop pack files.
+   See ./structure for the relation to SVN_FS_FS__FORMAT_NUMBER. */
+#define SVN_FS_FS__REVPROP_FORMAT_NUMBER   6
+
 /* Private FSFS-specific data shared between all svn_txn_t objects that
    relate to a particular transaction in a filesystem (as identified
    by transaction id and filesystem UUID).  Objects of this type are

Modified: subversion/branches/revprop-packing/subversion/libsvn_fs_fs/fs_fs.c
URL: http://svn.apache.org/viewvc/subversion/branches/revprop-packing/subversion/libsvn_fs_fs/fs_fs.c?rev=1163441&r1=1163440&r2=1163441&view=diff
==============================================================================
--- subversion/branches/revprop-packing/subversion/libsvn_fs_fs/fs_fs.c (original)
+++ subversion/branches/revprop-packing/subversion/libsvn_fs_fs/fs_fs.c Tue Aug 30 22:43:43
2011
@@ -1113,6 +1113,22 @@ check_format(int format)
      SVN_FS_FS__FORMAT_NUMBER, format);
 }
 
+static svn_error_t *
+check_revprop_format(svn_fs_t *fs, int format)
+{
+  fs_fs_data_t *ffd = fs->fsap_data;
+
+  if (ffd->format != SVN_FS_FS__MIN_PACKED_REVPROP_FORMAT
+      || format != SVN_FS_FS__REVPROP_FORMAT_NUMBER)
+    return svn_error_createf(SVN_ERR_FS_UNSUPPORTED_FORMAT, NULL,
+                             _("Found unsupported combination: "
+                               "revprop format '%d' and FSFS format %d"),
+                             /* ### TODO: add a FAQ URL to the error message? */
+                             format, ffd->format);
+
+  return SVN_NO_ERROR;
+}
+
 svn_boolean_t
 svn_fs_fs__fs_supports_mergeinfo(svn_fs_t *fs)
 {
@@ -3070,8 +3086,32 @@ read_revprop_manifest_record(apr_off_t *
   return SVN_NO_ERROR;
 }
 
+static svn_error_t *
+copy_first_line(svn_stream_t *from, svn_stream_t *to, apr_pool_t *pool)
+{
+  svn_stringbuf_t *format;
+  svn_boolean_t eof;
+
+  /* ### Use a 'goto' to keep the non-error codepath readable. */
+  SVN_ERR(svn_stream_readline(from, &format, "\n", &eof, pool));
+  if (eof) goto end;
+  svn_stringbuf_appendbyte(format, '\n');
+  SVN_ERR(svn_stream_write(to, format->data, &format->len));
+
+  return SVN_NO_ERROR;
+
+end:
+  return svn_error_createf(SVN_ERR_FS_CORRUPT, NULL,
+                           _("Format line vanished from packed revprop "
+                             "file (while write-lock is held)"));
+}
+
 /* Set the revision property list of revision REV in filesystem FS to
-   PROPLIST.  Use POOL for temporary allocations. */
+   PROPLIST.  Use POOL for temporary allocations.
+ 
+   NOTE: The implementation for packed revprops assumes that the caller has
+   a write lock and has checked the format number of the pre-existing revprops
+   pack file. */
 static svn_error_t *
 set_revision_proplist(svn_fs_t *fs,
                       svn_revnum_t rev,
@@ -3160,6 +3200,11 @@ set_revision_proplist(svn_fs_t *fs,
       SVN_ERR(svn_stream_open_readonly(&source_stream, pack_file_path,
                                        pool, pool));
 
+      /* Copy the format number. */
+      /* Note: not checking the revprop pack file format number because our
+         (one and only) caller did that. */
+      SVN_ERR(copy_first_line(source_stream, target_stream, pool));
+
       /* Copy manifest info up to the new prop's offset value. */
       SVN_ERR(svn_stream_bounded_copy(svn_stream_disown(source_stream, pool),
                                       svn_stream_disown(target_stream, pool),
@@ -3306,6 +3351,7 @@ revision_proplist(apr_hash_t **proplist_
       const char *pack_file_path;
       apr_off_t offset;
       apr_off_t manifest_record;
+      apr_size_t start; /* offset to first byte of the manifest */
 
       proplist = apr_hash_make(pool);
 
@@ -3315,10 +3361,24 @@ revision_proplist(apr_hash_t **proplist_
       /* Compute paths. */
       pack_file_path = path_revprops_pack(fs, rev, pool);
 
-      /* Open the pack file and seek to the manifest offset. */
+      /* Open the pack file and check the format number. */
       SVN_ERR(svn_io_file_open(&pack_file, pack_file_path,
                                APR_READ | APR_BUFFERED, APR_OS_DEFAULT, pool));
-      offset = shard_pos * (REVPROP_MANIFEST_FIELD_WIDTH + 1);
+      {
+        char buf[80];
+        int format;
+        apr_size_t len = sizeof(buf);
+
+        SVN_ERR(svn_io_read_length_line(pack_file, buf, &len, pool));
+        SVN_ERR(check_format_file_buffer_numeric(buf, 0, pack_file_path, pool));
+        SVN_ERR(svn_cstring_atoi(&format, buf));
+        SVN_ERR(check_revprop_format(fs, format));
+
+        start = len + 1; /* count the trailing newline */
+      }
+
+      /* Seek to the manifest record offset. */
+      offset = start + shard_pos * (REVPROP_MANIFEST_FIELD_WIDTH + 1);
       SVN_ERR(svn_io_file_seek(pack_file, APR_SET, &offset, pool));
 
       /* Read the revprop offset. */
@@ -3328,7 +3388,8 @@ revision_proplist(apr_hash_t **proplist_
                                                                     pool)));
 
       /* Seek to the revprop offset, and read the props. */
-      offset = ffd->max_files_per_dir * (REVPROP_MANIFEST_FIELD_WIDTH + 1)
+      offset = start
+               + ffd->max_files_per_dir * (REVPROP_MANIFEST_FIELD_WIDTH + 1)
                + manifest_record;
       SVN_ERR(svn_io_file_seek(pack_file, APR_SET, &offset, pool));
       SVN_ERR(svn_hash_read2(proplist,
@@ -8059,6 +8120,10 @@ pack_revprop_shard(svn_fs_t *fs,
   SVN_ERR(svn_stream_open_writable(&manifest_stream, manifest_file_path,
                                    pool, pool));
 
+  /* Staple the pack file with a format number. */
+  SVN_ERR(svn_stream_printf(manifest_stream, pool, "%d\n",
+                            SVN_FS_FS__REVPROP_FORMAT_NUMBER));
+
   start_rev = (svn_revnum_t) (shard * max_files_per_dir);
   end_rev = (svn_revnum_t) ((shard + 1) * (max_files_per_dir) - 1);
   next_offset = 0;

Modified: subversion/branches/revprop-packing/subversion/libsvn_fs_fs/structure
URL: http://svn.apache.org/viewvc/subversion/branches/revprop-packing/subversion/libsvn_fs_fs/structure?rev=1163441&r1=1163440&r2=1163441&view=diff
==============================================================================
--- subversion/branches/revprop-packing/subversion/libsvn_fs_fs/structure (original)
+++ subversion/branches/revprop-packing/subversion/libsvn_fs_fs/structure Tue Aug 30 22:43:43
2011
@@ -249,10 +249,14 @@ consumers that propedit r0.
 
 The pack+manifest file consists of:
 
+* A format number.
 * A 32-bit sequence number
 * A list of offsets, one for each revision's hunk in the pack file.
 * The concatenated hunks for all revisions in the shard.
 
+The format number is stored in ASCII decimal, followed by newline.  The only
+defined value is '6', understood by FSFS format 6+.
+
 The sequence number and the offsets are stored as ASCII decimal, followed by
 newline.  They are space-padded from the right such that the least-significant
 digit is on the tenth byte of the line for the sequence number, and on the
@@ -267,6 +271,7 @@ the shard.
 
 Readers operate as follows:
 
+* They read and check the format number.
 * They ignore the sequence number.
 * For r0, they always read revprops/0/0 read the non-packed file
   (e.g., revprops/0/0 if sharding is in use).
@@ -286,6 +291,8 @@ Editing a packed revision's properties p
 
 * Lock the revprop shard.  (via ./revprops/42.revpack/lock)
 * Prepare a revprop pack file, with an incremented sequence number.
+  (This is also an opportunity to update to a newer pack file format number,
+  once a non-'6' pack file format is defined.)
 * Lock the filesystem.  (via ./write-lock)
 * Check the sequence number of the revprop pack file.
 * Atomically move-into-place the new pack file.
@@ -294,6 +301,8 @@ Editing a packed revision's properties p
 
 This has the following properties:
 
+* Has a format number.
+
 * Having a per-shard lock minimizes writers' I/O.  (They have to rewrite
   a full shard file.)
 



Mime
View raw message