subversion-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sbut...@apache.org
Subject svn commit: r1351009 - in /subversion/trunk/subversion: include/ libsvn_repos/ svndumpfilter/ svnrdump/ tests/cmdline/ tests/cmdline/svndumpfilter_tests_data/
Date Sat, 16 Jun 2012 21:15:22 GMT
Author: sbutler
Date: Sat Jun 16 21:15:22 2012
New Revision: 1351009

URL: http://svn.apache.org/viewvc?rev=1351009&view=rev
Log:
Accept dump format v3, except for property deltas, in svndumpfilter.

Replace the hardcoded version number ("2") with a parser callback for
the dump file format version number.  Used by svndumpfilter only (so
far).

Add an API option to treat text-deltas as text instead of applying
them.  TRUE for svndumpfilter only.

* subversion/tests/cmdline/svndumpfilter_tests.py
  (with_deltas): New test.
  (test_list): Add the test.

* subversion/tests/cmdline/svndumpfilter_tests_data/simple_v3.dump
  New file.

* subversion/include/svn_repos.h
  (magic_header_record): New callback declaration.
  (svn_repos_parse_fns3_t): Add the new callback declaration.
  (svn_repos_parse_dumpstream3): Add the text-delta option.

* subversion/libsvn_repos/load.c
  (parse_format_version):  While we're here, put the "output" argument
   first.
  (svn_repos_parse_dumpstream3): Implement the text-delta-as-text option.
   Pass the format version to the new callback.  Don't reject v3 dumpfiles
   anymore.

* subversion/svndumpfilter/main.c
  (magic_header_record): New callback.
  (svn_repos_parse_fns3_t): Add the new callback.
  (parse_baton_initialize): Remove the hard-coded format-version output.
  (do_filter): Treat text-deltas as text.

* subversion/svnrdump/load_editor.c
  (magic_header_record): New callback.
  (svn_rdump__load_dumpstream): Use the new callback.

* subversion/libsvn_repos/deprecated.c
  (fns3_from_fns2): Set the new callback.
  (svn_repos_get_fs_build_parser3): Add the text-delta option.

* subversion/libsvn_repos/load-fs-vtable.c
  (magic_header_record): New callback.
  (svn_repos_get_fs_build_parser4): Use the new callback.
  (svn_repos_load_fs4): Add the text-delta option.

Added:
    subversion/trunk/subversion/tests/cmdline/svndumpfilter_tests_data/simple_v3.dump
Modified:
    subversion/trunk/subversion/include/svn_repos.h
    subversion/trunk/subversion/libsvn_repos/deprecated.c
    subversion/trunk/subversion/libsvn_repos/load-fs-vtable.c
    subversion/trunk/subversion/libsvn_repos/load.c
    subversion/trunk/subversion/svndumpfilter/main.c
    subversion/trunk/subversion/svnrdump/load_editor.c
    subversion/trunk/subversion/tests/cmdline/svndumpfilter_tests.py

Modified: subversion/trunk/subversion/include/svn_repos.h
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/include/svn_repos.h?rev=1351009&r1=1351008&r2=1351009&view=diff
==============================================================================
--- subversion/trunk/subversion/include/svn_repos.h (original)
+++ subversion/trunk/subversion/include/svn_repos.h Sat Jun 16 21:15:22 2012
@@ -2706,12 +2706,19 @@ svn_repos_load_fs(svn_repos_t *repos,
 
 /**
  * A vtable that is driven by svn_repos_parse_dumpstream3().
- * ### TODO: Add a new callback for the dumpfile version header.
  *
  * @since New in 1.8.
  */
 typedef struct svn_repos_parse_fns3_t
 {
+  /** The parser has discovered a new "magic header" record within the
+   * parsing session represented by @a parse_baton.  The dump-format
+   * version number is @a version.
+   */
+  svn_error_t *(*magic_header_record)(int version,
+                                      void *parse_baton,
+                                      apr_pool_t *pool);
+
   /** The parser has discovered a new uuid record within the parsing
    * session represented by @a parse_baton.  The uuid's value is
    * @a uuid, and it is allocated in @a pool.
@@ -2801,8 +2808,10 @@ typedef struct svn_repos_parse_fns3_t
  * Read and parse dumpfile-formatted @a stream, calling callbacks in
  * @a parse_fns/@a parse_baton, and using @a pool for allocations.
  *
- * ### TODO: Add a boolean option to treat text-deltas as text, because
- * a dump-filtering tool shouldn't apply the deltas.
+ * If @a deltas_are_text is @c TRUE, handle text-deltas with the @a
+ * set_fulltext callback.  This is useful when manipulating a dump
+ * stream without loading it.  Otherwise handle text-deltas with the
+ * @a apply_textdelta callback.
  *
  * If @a cancel_func is not @c NULL, it is called periodically with
  * @a cancel_baton as argument to see if the client wishes to cancel
@@ -2811,7 +2820,7 @@ typedef struct svn_repos_parse_fns3_t
  * This parser has built-in knowledge of the dumpfile format, but only
  * in a limited sense:
  *
- *    * ### TODO: it recognizes the "magic" format-version header.
+ *    * it recognizes the "magic" format-version header.
  *
  *    * it recognizes the UUID header.
  *
@@ -2825,7 +2834,8 @@ typedef struct svn_repos_parse_fns3_t
  *      and text, and pass the pieces to the vtable.
  *
  * This is enough knowledge to make it easy on vtable implementors,
- * but still allow expansion of the format:  most headers are ignored.
+ * but still allow expansion of the format: most headers do not have
+ * to be handled explicitly.
  *
  * @since New in 1.8.
  */
@@ -2833,6 +2843,7 @@ svn_error_t *
 svn_repos_parse_dumpstream3(svn_stream_t *stream,
                             const svn_repos_parse_fns3_t *parse_fns,
                             void *parse_baton,
+                            svn_boolean_t deltas_are_text,
                             svn_cancel_func_t cancel_func,
                             void *cancel_baton,
                             apr_pool_t *pool);

Modified: subversion/trunk/subversion/libsvn_repos/deprecated.c
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_repos/deprecated.c?rev=1351009&r1=1351008&r2=1351009&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_repos/deprecated.c (original)
+++ subversion/trunk/subversion/libsvn_repos/deprecated.c Sat Jun 16 21:15:22 2012
@@ -824,6 +824,7 @@ fns3_from_fns2(const svn_repos_parser_fn
   svn_repos_parse_fns3_t *fns3;
 
   fns3 = apr_palloc(pool, sizeof(*fns3));
+  fns3->magic_header_record = NULL;
   fns3->uuid_record = fns2->uuid_record;
   fns3->new_revision_record = fns2->new_revision_record;
   fns3->new_node_record = fns2->new_node_record;
@@ -848,7 +849,7 @@ svn_repos_parse_dumpstream2(svn_stream_t
 {
   svn_repos_parse_fns3_t *fns3 = fns3_from_fns2(parse_fns, pool);
 
-  return svn_repos_parse_dumpstream3(stream, fns3, parse_baton,
+  return svn_repos_parse_dumpstream3(stream, fns3, parse_baton, FALSE,
                                      cancel_func, cancel_baton, pool);
 }
 

Modified: subversion/trunk/subversion/libsvn_repos/load-fs-vtable.c
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_repos/load-fs-vtable.c?rev=1351009&r1=1351008&r2=1351009&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_repos/load-fs-vtable.c (original)
+++ subversion/trunk/subversion/libsvn_repos/load-fs-vtable.c Sat Jun 16 21:15:22 2012
@@ -589,6 +589,13 @@ maybe_add_with_history(struct node_baton
   return SVN_NO_ERROR;
 }
 
+static svn_error_t *
+magic_header_record(int version,
+                    void *parse_baton,
+                    apr_pool_t *pool)
+{
+  return SVN_NO_ERROR;
+}
 
 static svn_error_t *
 uuid_record(const char *uuid,
@@ -1042,6 +1049,7 @@ svn_repos_get_fs_build_parser4(const svn
   if (SVN_IS_VALID_REVNUM(start_rev))
     SVN_ERR_ASSERT(start_rev <= end_rev);
 
+  parser->magic_header_record = magic_header_record;
   parser->uuid_record = uuid_record;
   parser->new_revision_record = new_revision_record;
   parser->new_node_record = new_node_record;
@@ -1116,6 +1124,6 @@ svn_repos_load_fs4(svn_repos_t *repos,
   pb->use_pre_commit_hook = use_pre_commit_hook;
   pb->use_post_commit_hook = use_post_commit_hook;
 
-  return svn_repos_parse_dumpstream3(dumpstream, parser, parse_baton,
+  return svn_repos_parse_dumpstream3(dumpstream, parser, parse_baton, FALSE,
                                      cancel_func, cancel_baton, pool);
 }

Modified: subversion/trunk/subversion/libsvn_repos/load.c
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/libsvn_repos/load.c?rev=1351009&r1=1351008&r2=1351009&view=diff
==============================================================================
--- subversion/trunk/subversion/libsvn_repos/load.c (original)
+++ subversion/trunk/subversion/libsvn_repos/load.c Sat Jun 16 21:15:22 2012
@@ -373,7 +373,8 @@ parse_text_block(svn_stream_t *stream,
 /* Parse VERSIONSTRING and verify that we support the dumpfile format
    version number, setting *VERSION appropriately. */
 static svn_error_t *
-parse_format_version(const char *versionstring, int *version)
+parse_format_version(int *version,
+                     const char *versionstring)
 {
   static const int magic_len = sizeof(SVN_REPOS_DUMPFILE_MAGIC_HEADER) - 1;
   const char *p = strchr(versionstring, ':');
@@ -409,6 +410,7 @@ svn_error_t *
 svn_repos_parse_dumpstream3(svn_stream_t *stream,
                             const svn_repos_parse_fns3_t *parse_fns,
                             void *parse_baton,
+                            svn_boolean_t deltas_are_text,
                             svn_cancel_func_t cancel_func,
                             void *cancel_baton,
                             apr_pool_t *pool)
@@ -428,16 +430,11 @@ svn_repos_parse_dumpstream3(svn_stream_t
     return stream_ran_dry();
 
   /* The first two lines of the stream are the dumpfile-format version
-     number, and a blank line. */
-  SVN_ERR(parse_format_version(linebuf->data, &version));
-
-  /* If we were called from svn_repos_parse_dumpstream(), the
-     callbacks to handle delta contents will be NULL, so we have to
-     reject dumpfiles with the current version. */
-  if (version == SVN_REPOS_DUMPFILE_FORMAT_VERSION
-      && (!parse_fns->delete_node_property || !parse_fns->apply_textdelta))
-    return svn_error_createf(SVN_ERR_STREAM_MALFORMED_DATA, NULL,
-                             _("Unsupported dumpfile version: %d"), version);
+     number, and a blank line.  To preserve backward compatibility,
+     don't assume the existence of newer parser-vtable functions. */
+  SVN_ERR(parse_format_version(&version, linebuf->data));
+  if (parse_fns->magic_header_record != NULL)
+    SVN_ERR(parse_fns->magic_header_record(version, parse_baton, pool));
 
   /* A dumpfile "record" is defined to be a header-block of
      rfc822-style headers, possibly followed by a content-block.
@@ -528,6 +525,7 @@ svn_repos_parse_dumpstream3(svn_stream_t
           SVN_ERR(parse_fns->uuid_record(value, parse_baton, pool));
         }
       /* Or perhaps a dumpfile format? */
+      /* ### TODO: use parse_format_version */
       else if ((value = apr_hash_get(headers,
                                      SVN_REPOS_DUMPFILE_MAGIC_HEADER,
                                      APR_HASH_KEY_STRING)))
@@ -591,7 +589,9 @@ svn_repos_parse_dumpstream3(svn_stream_t
           const char *delta = apr_hash_get(headers,
                                            SVN_REPOS_DUMPFILE_TEXT_DELTA,
                                            APR_HASH_KEY_STRING);
-          svn_boolean_t is_delta = (delta && strcmp(delta, "true") == 0);
+          svn_boolean_t is_delta = FALSE;
+          if (! deltas_are_text)
+            is_delta = (delta && strcmp(delta, "true") == 0);
 
           SVN_ERR(parse_text_block(stream,
                                    svn__atoui64(text_cl),

Modified: subversion/trunk/subversion/svndumpfilter/main.c
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/svndumpfilter/main.c?rev=1351009&r1=1351008&r2=1351009&view=diff
==============================================================================
--- subversion/trunk/subversion/svndumpfilter/main.c (original)
+++ subversion/trunk/subversion/svndumpfilter/main.c Sat Jun 16 21:15:22 2012
@@ -256,6 +256,19 @@ struct node_baton_t
 
 /* Filtering vtable members */
 
+/* File-format stamp. */
+static svn_error_t *
+magic_header_record(int version, void *parse_baton, apr_pool_t *pool)
+{
+  struct parse_baton_t *pb = parse_baton;
+  SVN_ERR(svn_stream_printf(pb->out_stream, pool,
+                            SVN_REPOS_DUMPFILE_MAGIC_HEADER ": %d\n\n",
+                            version));
+
+  return SVN_NO_ERROR;
+}
+
+
 /* New revision: set up revision_baton, decide if we skip it. */
 static svn_error_t *
 new_revision_record(void **revision_baton,
@@ -880,6 +893,7 @@ close_revision(void *revision_baton)
 /* Filtering vtable */
 svn_repos_parse_fns3_t filtering_vtable =
   {
+    magic_header_record,
     uuid_record,
     new_revision_record,
     new_node_record,
@@ -1032,17 +1046,6 @@ parse_baton_initialize(struct parse_bato
   baton->last_live_revision = SVN_INVALID_REVNUM;
   baton->oldest_original_rev = SVN_INVALID_REVNUM;
 
-  /* This is non-ideal: We should pass through the version of the
-   * input dumpstream.  However, our API currently doesn't allow that.
-   * Hardcoding version 2 is acceptable because:
-   *   - We currently do not accept version 3 or greater.
-   *   - Dumpstream version 1 is so ancient as to be ignorable
-   *     (0.17.x and earlier)
-   */
-  SVN_ERR(svn_stream_printf(baton->out_stream, pool,
-                            SVN_REPOS_DUMPFILE_MAGIC_HEADER ": %d\n\n",
-                            2));
-
   *pb = baton;
   return SVN_NO_ERROR;
 }
@@ -1145,7 +1148,7 @@ do_filter(apr_getopt_t *os,
 
   SVN_ERR(parse_baton_initialize(&pb, opt_state, do_exclude, pool));
   SVN_ERR(svn_repos_parse_dumpstream3(pb->in_stream, &filtering_vtable, pb,
-                                      NULL, NULL, pool));
+                                      TRUE, NULL, NULL, pool));
 
   /* The rest of this is just reporting.  If we aren't reporting, get
      outta here. */

Modified: subversion/trunk/subversion/svnrdump/load_editor.c
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/svnrdump/load_editor.c?rev=1351009&r1=1351008&r2=1351009&view=diff
==============================================================================
--- subversion/trunk/subversion/svnrdump/load_editor.c (original)
+++ subversion/trunk/subversion/svnrdump/load_editor.c Sat Jun 16 21:15:22 2012
@@ -584,6 +584,14 @@ new_revision_record(void **revision_bato
 }
 
 static svn_error_t *
+magic_header_record(int version,
+            void *parse_baton,
+            apr_pool_t *pool)
+{
+  return SVN_NO_ERROR;
+}
+
+static svn_error_t *
 uuid_record(const char *uuid,
             void *parse_baton,
             apr_pool_t *pool)
@@ -1163,8 +1171,9 @@ svn_rdump__load_dumpstream(svn_stream_t 
                                            session_url, pool));
 
   parser = apr_pcalloc(pool, sizeof(*parser));
-  parser->new_revision_record = new_revision_record;
+  parser->magic_header_record = magic_header_record;
   parser->uuid_record = uuid_record;
+  parser->new_revision_record = new_revision_record;
   parser->new_node_record = new_node_record;
   parser->set_revision_property = set_revision_property;
   parser->set_node_property = set_node_property;
@@ -1185,7 +1194,7 @@ svn_rdump__load_dumpstream(svn_stream_t 
   parse_baton->last_rev_mapped = SVN_INVALID_REVNUM;
   parse_baton->oldest_dumpstream_rev = SVN_INVALID_REVNUM;
 
-  err = svn_repos_parse_dumpstream3(stream, parser, parse_baton,
+  err = svn_repos_parse_dumpstream3(stream, parser, parse_baton, FALSE,
                                     cancel_func, cancel_baton, pool);
 
   /* If all goes well, or if we're cancelled cleanly, don't leave a

Modified: subversion/trunk/subversion/tests/cmdline/svndumpfilter_tests.py
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/cmdline/svndumpfilter_tests.py?rev=1351009&r1=1351008&r2=1351009&view=diff
==============================================================================
--- subversion/trunk/subversion/tests/cmdline/svndumpfilter_tests.py (original)
+++ subversion/trunk/subversion/tests/cmdline/svndumpfilter_tests.py Sat Jun 16 21:15:22 2012
@@ -33,8 +33,9 @@ import tempfile
 import svntest
 from svntest.verify import SVNExpectedStdout, SVNExpectedStderr
 
-# Get some helper routines from svnadmin_tests
+# Get some helper routines
 from svnadmin_tests import load_and_verify_dumpstream, test_create
+from svntest.main import run_svn, run_svnadmin
 
 # (abbreviation)
 Skip = svntest.testcase.Skip_deco
@@ -651,6 +652,23 @@ def match_empty_prefix(sbox):
   # doesn't seem to be a consistent way to quote such an argument to
   # prevent expansion.
 
+@Issue(2760)
+def accepts_deltas(sbox):
+  "accepts deltas in the input"
+  # Accept format v3 (as created by 'svnadmin --deltas' or svnrdump).
+
+  test_create(sbox)
+  dumpfile_location = os.path.join(os.path.dirname(sys.argv[0]),
+                                   'svndumpfilter_tests_data',
+                                   'simple_v3.dump')
+  dumpfile = open(dumpfile_location).read()
+
+  filtered_out, filtered_err = filter_and_return_output(dumpfile, 0, "include",
+                                                        "trunk", "--quiet")
+  load_and_verify_dumpstream(sbox, [], [], None, filtered_out)
+
+  
+
 ########################################################################
 # Run the tests
 
@@ -664,6 +682,7 @@ test_list = [ None,
               filter_mergeinfo_revs_outside_of_dump_stream,
               dropped_but_not_renumbered_empty_revs,
               match_empty_prefix,
+              accepts_deltas,
               ]
 
 if __name__ == '__main__':

Added: subversion/trunk/subversion/tests/cmdline/svndumpfilter_tests_data/simple_v3.dump
URL: http://svn.apache.org/viewvc/subversion/trunk/subversion/tests/cmdline/svndumpfilter_tests_data/simple_v3.dump?rev=1351009&view=auto
==============================================================================
--- subversion/trunk/subversion/tests/cmdline/svndumpfilter_tests_data/simple_v3.dump (added)
+++ subversion/trunk/subversion/tests/cmdline/svndumpfilter_tests_data/simple_v3.dump Sat
Jun 16 21:15:22 2012
@@ -0,0 +1,71 @@
+SVN-fs-dump-format-version: 3
+
+UUID: 95ae2ea4-9df8-40f9-8aee-3e2af6f6aa3c
+
+Revision-number: 0
+Prop-content-length: 56
+Content-length: 56
+
+K 8
+svn:date
+V 27
+2012-06-15T15:58:51.002886Z
+PROPS-END
+
+Revision-number: 1
+Prop-content-length: 103
+Content-length: 103
+
+K 10
+svn:author
+V 7
+jrandom
+K 8
+svn:date
+V 27
+2012-06-15T16:00:19.178493Z
+K 7
+svn:log
+V 4
+Test
+PROPS-END
+
+Node-path: trunk
+Node-kind: dir
+Node-action: add
+Prop-content-length: 41
+Content-length: 41
+
+K 4
+soup
+V 16
+No soup for you.
+PROPS-END
+
+
+Revision-number: 2
+Prop-content-length: 114
+Content-length: 114
+
+K 10
+svn:author
+V 7
+jrandom
+K 8
+svn:date
+V 27
+2012-06-15T16:01:26.311580Z
+K 7
+svn:log
+V 14
+Create branch
+
+PROPS-END
+
+Node-path: branch1
+Node-kind: dir
+Node-action: add
+Node-copyfrom-rev: 1
+Node-copyfrom-path: trunk
+
+



Mime
View raw message