Return-Path: X-Original-To: apmail-subversion-commits-archive@minotaur.apache.org Delivered-To: apmail-subversion-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E9B2210B59 for ; Wed, 24 Jul 2013 16:46:19 +0000 (UTC) Received: (qmail 69756 invoked by uid 500); 24 Jul 2013 16:46:19 -0000 Delivered-To: apmail-subversion-commits-archive@subversion.apache.org Received: (qmail 69644 invoked by uid 500); 24 Jul 2013 16:46:18 -0000 Mailing-List: contact commits-help@subversion.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@subversion.apache.org Delivered-To: mailing list commits@subversion.apache.org Received: (qmail 69636 invoked by uid 99); 24 Jul 2013 16:46:17 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 24 Jul 2013 16:46:17 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=5.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.4] (HELO eris.apache.org) (140.211.11.4) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 24 Jul 2013 16:46:11 +0000 Received: from eris.apache.org (localhost [127.0.0.1]) by eris.apache.org (Postfix) with ESMTP id 8034A2388900; Wed, 24 Jul 2013 16:45:49 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1506622 - in /subversion/branches/fsfs-improvements: subversion/libsvn_fs_fs/ tools/server-side/ Date: Wed, 24 Jul 2013 16:45:49 -0000 To: commits@subversion.apache.org From: stefan2@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20130724164549.8034A2388900@eris.apache.org> X-Virus-Checked: Checked by ClamAV on apache.org Author: stefan2 Date: Wed Jul 24 16:45:48 2013 New Revision: 1506622 URL: http://svn.apache.org/r1506622 Log: On the fsfs-improvements branch: Flatten the representation_t structure by replacing the checksums sub-structures with directly embedded digests. That ensures type safety and simplifies access, checks and copies. * subversion/libsvn_fs_fs/fs.h (representation_t): use digests and a flag instead of pool-allocated checksum objects * subversion/libsvn_fs_fs/low_level.h (svn_fs_fs__unparse_representation): drop the maybe_corrupt flag as there are no sub-structures anymore * subversion/libsvn_fs_fs/low_level.c (svn_fs_fs__parse_representation): update parser (format_digest): new utility replacing DISPLAY_MAYBE_NULL_CHECKSUM (DISPLAY_MAYBE_NULL_CHECKSUM): drop (svn_fs_fs__unparse_representation): update serializer (svn_fs_fs__write_noderev): update caller * subversion/libsvn_fs_fs/cached_data.c (create_rep_state): update caller (rep_read_baton, delta_read_baton): use digest instead of svn_checksum_t (rep_read_get_baton, rep_read_contents, svn_fs_fs__get_file_delta_stream): update struct users (delta_read_md5_digest): greatly simplify * subversion/libsvn_fs_fs/fs_fs.c (svn_fs_fs__file_checksum): update struct user (svn_fs_fs__rep_copy): greatly simplify * subversion/libsvn_fs_fs/rep-cache.c (svn_fs_fs__walk_rep_reference, svn_fs_fs__get_rep_reference, svn_fs_fs__set_rep_reference): update struct users * subversion/libsvn_fs_fs/temp_serializer.c (serialize_checksum, deserialize_checksum): drop because no longer needed (serialize_representation, svn_fs_fs__noderev_deserialize): simplify as representation_t no longer contains references to other structs * subversion/libsvn_fs_fs/transaction.c (path_txn_sha1): change parameter from checksum to digest; update code (store_sha1_rep_mapping, get_shared_rep): update struct user (digests_final): new utility to extract the digests from checksums (rep_write_contents_close, write_hash_rep, write_hash_delta_rep): use the new utility to init representation_t (write_final_rev): update struct user * tools/server-side/svn-rep-sharing-stats.c (value_t): use digest instead of svn_checksum_t (record, pretty_print): update struct users Modified: subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/cached_data.c subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.h subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs_fs.c subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/low_level.c subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/low_level.h subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/rep-cache.c subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/temp_serializer.c subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/transaction.c subversion/branches/fsfs-improvements/tools/server-side/svn-rep-sharing-stats.c Modified: subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/cached_data.c URL: http://svn.apache.org/viewvc/subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/cached_data.c?rev=1506622&r1=1506621&r2=1506622&view=diff ============================================================================== --- subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/cached_data.c (original) +++ subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/cached_data.c Wed Jul 24 16:45:48 2013 @@ -687,7 +687,7 @@ create_rep_state(rep_state_t **rep_state "Corrupt representation '%s'", rep ? svn_fs_fs__unparse_representation - (rep, ffd->format, TRUE, TRUE, pool)->data + (rep, ffd->format, TRUE, pool)->data : "(null)"); } /* ### Call representation_string() ? */ @@ -796,7 +796,7 @@ struct rep_read_baton length, and the amount we've read so far. Some of this information is redundant with rs_list and src_state, but it's convenient for the checksumming code to have it here. */ - svn_checksum_t *md5_checksum; + unsigned char md5_digest[APR_MD5_DIGESTSIZE]; svn_filesize_t len; svn_filesize_t off; @@ -1100,7 +1100,7 @@ rep_read_get_baton(struct rep_read_baton b->buf = NULL; b->md5_checksum_ctx = svn_checksum_ctx_create(svn_checksum_md5, pool); b->checksum_finalized = FALSE; - b->md5_checksum = svn_checksum_dup(rep->md5_checksum, pool); + memcpy(b->md5_digest, rep->md5_digest, sizeof(rep->md5_digest)); b->len = rep->expanded_size; b->off = 0; b->fulltext_cache_key = fulltext_cache_key; @@ -1457,13 +1457,16 @@ rep_read_contents(void *baton, if (rb->off == rb->len) { svn_checksum_t *md5_checksum; + svn_checksum_t expected; + expected.kind = svn_checksum_md5; + expected.digest = rb->md5_digest; rb->checksum_finalized = TRUE; SVN_ERR(svn_checksum_final(&md5_checksum, rb->md5_checksum_ctx, rb->pool)); - if (!svn_checksum_match(md5_checksum, rb->md5_checksum)) + if (!svn_checksum_match(md5_checksum, &expected)) return svn_error_create(SVN_ERR_FS_CORRUPT, - svn_checksum_mismatch_err(rb->md5_checksum, md5_checksum, + svn_checksum_mismatch_err(&expected, md5_checksum, rb->pool, _("Checksum mismatch while reading representation")), NULL); @@ -1601,7 +1604,7 @@ svn_fs_fs__try_process_file_contents(svn struct delta_read_baton { rep_state_t *rs; - svn_checksum_t *checksum; + unsigned char md5_digest[APR_MD5_DIGESTSIZE]; }; /* This implements the svn_txdelta_next_window_fn_t interface. */ @@ -1625,11 +1628,7 @@ static const unsigned char * delta_read_md5_digest(void *baton) { struct delta_read_baton *drb = baton; - - if (drb->checksum->kind == svn_checksum_md5) - return drb->checksum->digest; - else - return NULL; + return drb->md5_digest; } svn_error_t * @@ -1660,8 +1659,8 @@ svn_fs_fs__get_file_delta_stream(svn_txd /* Create the delta read baton. */ struct delta_read_baton *drb = apr_pcalloc(pool, sizeof(*drb)); drb->rs = rep_state; - drb->checksum = svn_checksum_dup(target->data_rep->md5_checksum, - pool); + memcpy(drb->md5_digest, target->data_rep->md5_digest, + sizeof(drb->md5_digest)); *stream_p = svn_txdelta_stream_create(drb, delta_read_next_window, delta_read_md5_digest, pool); return SVN_NO_ERROR; Modified: subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.h URL: http://svn.apache.org/viewvc/subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.h?rev=1506622&r1=1506621&r2=1506622&view=diff ============================================================================== --- subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.h (original) +++ subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.h Wed Jul 24 16:45:48 2013 @@ -26,6 +26,8 @@ #include #include #include +#include +#include #include "svn_fs.h" #include "svn_config.h" @@ -396,20 +398,20 @@ typedef struct transaction_t * svn_fs_fs__rep_copy. */ typedef struct representation_t { - /* Checksums for the contents produced by this representation. + /* Checksums digests for the contents produced by this representation. This checksum is for the contents the rep shows to consumers, regardless of how the rep stores the data under the hood. It is independent of the storage (fulltext, delta, whatever). - If checksum is NULL, then for compatibility behave as though this + If has_sha1 is FALSE, then for compatibility behave as though this checksum matches the expected checksum. The md5 checksum is always filled, unless this is rep which was retrieved from the rep-cache. The sha1 checksum is only computed on - a write, for use with rep-sharing; it may be read from an existing - representation, but otherwise it is NULL. */ - svn_checksum_t *md5_checksum; - svn_checksum_t *sha1_checksum; + a write, for use with rep-sharing. */ + svn_boolean_t has_sha1; + unsigned char sha1_digest[APR_SHA1_DIGESTSIZE]; + unsigned char md5_digest[APR_MD5_DIGESTSIZE]; /* Revision where this representation is located. */ svn_revnum_t revision; Modified: subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs_fs.c URL: http://svn.apache.org/viewvc/subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs_fs.c?rev=1506622&r1=1506621&r2=1506622&view=diff ============================================================================== --- subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs_fs.c (original) +++ subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs_fs.c Wed Jul 24 16:45:48 2013 @@ -948,24 +948,32 @@ svn_fs_fs__file_checksum(svn_checksum_t svn_checksum_kind_t kind, apr_pool_t *pool) { + *checksum = NULL; + if (noderev->data_rep) { + svn_checksum_t temp; + temp.kind = kind; + switch(kind) { case svn_checksum_md5: - *checksum = svn_checksum_dup(noderev->data_rep->md5_checksum, - pool); + temp.digest = noderev->data_rep->md5_digest; break; + case svn_checksum_sha1: - *checksum = svn_checksum_dup(noderev->data_rep->sha1_checksum, - pool); + if (! noderev->data_rep->has_sha1) + return SVN_NO_ERROR; + + temp.digest = noderev->data_rep->sha1_digest; break; + default: - *checksum = NULL; + return SVN_NO_ERROR; } + + *checksum = svn_checksum_dup(&temp, pool); } - else - *checksum = NULL; return SVN_NO_ERROR; } @@ -974,18 +982,10 @@ representation_t * svn_fs_fs__rep_copy(representation_t *rep, apr_pool_t *pool) { - representation_t *rep_new; - if (rep == NULL) return NULL; - rep_new = apr_pcalloc(pool, sizeof(*rep_new)); - - memcpy(rep_new, rep, sizeof(*rep_new)); - rep_new->md5_checksum = svn_checksum_dup(rep->md5_checksum, pool); - rep_new->sha1_checksum = svn_checksum_dup(rep->sha1_checksum, pool); - - return rep_new; + return apr_pmemdup(pool, rep, sizeof(*rep)); } Modified: subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/low_level.c URL: http://svn.apache.org/viewvc/subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/low_level.c?rev=1506622&r1=1506621&r2=1506622&view=diff ============================================================================== --- subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/low_level.c (original) +++ subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/low_level.c Wed Jul 24 16:45:48 2013 @@ -20,9 +20,6 @@ * ==================================================================== */ -#include -#include - #include "svn_private_config.h" #include "svn_hash.h" #include "svn_pools.h" @@ -526,6 +523,7 @@ svn_fs_fs__parse_representation(represen char *str; apr_int64_t val; char *string = text->data; + svn_checksum_t *checksum; rep = apr_pcalloc(pool, sizeof(*rep)); *rep_p = rep; @@ -576,8 +574,8 @@ svn_fs_fs__parse_representation(represen return svn_error_create(SVN_ERR_FS_CORRUPT, NULL, _("Malformed text representation offset line in node-rev")); - SVN_ERR(svn_checksum_parse_hex(&rep->md5_checksum, svn_checksum_md5, str, - pool)); + SVN_ERR(svn_checksum_parse_hex(&checksum, svn_checksum_md5, str, pool)); + memcpy(rep->md5_digest, checksum->digest, sizeof(rep->md5_digest)); /* The remaining fields are only used for formats >= 4, so check that. */ str = svn_cstring_tokenize(" ", &string); @@ -589,8 +587,9 @@ svn_fs_fs__parse_representation(represen return svn_error_create(SVN_ERR_FS_CORRUPT, NULL, _("Malformed text representation offset line in node-rev")); - SVN_ERR(svn_checksum_parse_hex(&rep->sha1_checksum, svn_checksum_sha1, str, - pool)); + SVN_ERR(svn_checksum_parse_hex(&checksum, svn_checksum_sha1, str, pool)); + rep->has_sha1 = checksum != NULL; + memcpy(rep->sha1_digest, checksum->digest, sizeof(rep->sha1_digest)); /* Read the uniquifier. */ str = svn_cstring_tokenize("/", &string); @@ -794,40 +793,55 @@ svn_fs_fs__read_noderev(node_revision_t return SVN_NO_ERROR; } +/* Return a textual representation of the DIGEST of given KIND. + * If IS_NULL is TRUE, no digest is available. + * Use POOL for allocations. + */ +static const char * +format_digest(const unsigned char *digest, + svn_checksum_kind_t kind, + svn_boolean_t is_null, + apr_pool_t *pool) +{ + svn_checksum_t checksum; + checksum.digest = digest; + checksum.kind = kind; + + if (is_null) + return "(null)"; + + return svn_checksum_to_cstring_display(&checksum, pool); +} + svn_stringbuf_t * svn_fs_fs__unparse_representation(representation_t *rep, int format, svn_boolean_t mutable_rep_truncated, - svn_boolean_t may_be_corrupt, apr_pool_t *pool) { char buffer[SVN_INT64_BUFFER_SIZE]; if (svn_fs_fs__id_txn_used(&rep->txn_id) && mutable_rep_truncated) return svn_stringbuf_ncreate("-1", 2, pool); -#define DISPLAY_MAYBE_NULL_CHECKSUM(checksum) \ - ((!may_be_corrupt || (checksum) != NULL) \ - ? svn_checksum_to_cstring_display((checksum), pool) \ - : "(null)") - - if (format < SVN_FS_FS__MIN_REP_SHARING_FORMAT || rep->sha1_checksum == NULL) + if (format < SVN_FS_FS__MIN_REP_SHARING_FORMAT || !rep->has_sha1) return svn_stringbuf_createf - (pool, "%ld %" APR_OFF_T_FMT " %" SVN_FILESIZE_T_FMT - " %" SVN_FILESIZE_T_FMT " %s", - rep->revision, rep->offset, rep->size, - rep->expanded_size, - DISPLAY_MAYBE_NULL_CHECKSUM(rep->md5_checksum)); + (pool, "%ld %" APR_OFF_T_FMT " %" SVN_FILESIZE_T_FMT + " %" SVN_FILESIZE_T_FMT " %s", + rep->revision, rep->offset, rep->size, + rep->expanded_size, + format_digest(rep->md5_digest, svn_checksum_md5, FALSE, pool)); svn__ui64tobase36(buffer, rep->uniquifier.number); return svn_stringbuf_createf - (pool, "%ld %" APR_OFF_T_FMT " %" SVN_FILESIZE_T_FMT - " %" SVN_FILESIZE_T_FMT " %s %s %s/_%s", - rep->revision, rep->offset, rep->size, - rep->expanded_size, - DISPLAY_MAYBE_NULL_CHECKSUM(rep->md5_checksum), - DISPLAY_MAYBE_NULL_CHECKSUM(rep->sha1_checksum), - svn_fs_fs__id_txn_unparse(&rep->uniquifier.txn_id, pool), - buffer); + (pool, "%ld %" APR_OFF_T_FMT " %" SVN_FILESIZE_T_FMT + " %" SVN_FILESIZE_T_FMT " %s %s %s/_%s", + rep->revision, rep->offset, rep->size, + rep->expanded_size, + format_digest(rep->md5_digest, svn_checksum_md5, FALSE, pool), + format_digest(rep->sha1_digest, svn_checksum_sha1, + !rep->has_sha1, pool), + svn_fs_fs__id_txn_unparse(&rep->uniquifier.txn_id, pool), + buffer); #undef DISPLAY_MAYBE_NULL_CHECKSUM @@ -863,14 +877,13 @@ svn_fs_fs__write_noderev(svn_stream_t *o (noderev->data_rep, format, noderev->kind == svn_node_dir, - FALSE, pool)->data)); if (noderev->prop_rep) SVN_ERR(svn_stream_printf(outfile, pool, HEADER_PROPS ": %s\n", svn_fs_fs__unparse_representation (noderev->prop_rep, format, - TRUE, FALSE, pool)->data)); + TRUE, pool)->data)); SVN_ERR(svn_stream_printf(outfile, pool, HEADER_CPATH ": %s\n", noderev->created_path)); Modified: subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/low_level.h URL: http://svn.apache.org/viewvc/subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/low_level.h?rev=1506622&r1=1506621&r2=1506622&view=diff ============================================================================== --- subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/low_level.h (original) +++ subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/low_level.h Wed Jul 24 16:45:48 2013 @@ -115,7 +115,6 @@ svn_stringbuf_t * svn_fs_fs__unparse_representation(representation_t *rep, int format, svn_boolean_t mutable_rep_truncated, - svn_boolean_t may_be_corrupt, apr_pool_t *pool); /* This type enumerates all forms of representations that we support. */ Modified: subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/rep-cache.c URL: http://svn.apache.org/viewvc/subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/rep-cache.c?rev=1506622&r1=1506621&r2=1506622&view=diff ============================================================================== --- subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/rep-cache.c (original) +++ subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/rep-cache.c Wed Jul 24 16:45:48 2013 @@ -178,6 +178,7 @@ svn_fs_fs__walk_rep_reference(svn_fs_t * representation_t *rep; const char *sha1_digest; svn_error_t *err; + svn_checksum_t *checksum; /* Clear ITERPOOL occasionally. */ if (iterations++ % 16 == 0) @@ -195,11 +196,13 @@ svn_fs_fs__walk_rep_reference(svn_fs_t * rep = apr_pcalloc(iterpool, sizeof(*rep)); svn_fs_fs__id_txn_reset(&rep->txn_id); sha1_digest = svn_sqlite__column_text(stmt, 0, iterpool); - err = svn_checksum_parse_hex(&rep->sha1_checksum, - svn_checksum_sha1, sha1_digest, - iterpool); + err = svn_checksum_parse_hex(&checksum, svn_checksum_sha1, + sha1_digest, iterpool); if (err) return svn_error_compose_create(err, svn_sqlite__reset(stmt)); + + rep->has_sha1 = TRUE; + memcpy(rep->sha1_digest, checksum->digest, sizeof(rep->sha1_digest)); rep->revision = svn_sqlite__column_revnum(stmt, 1); rep->offset = svn_sqlite__column_int64(stmt, 2); rep->size = svn_sqlite__column_int64(stmt, 3); @@ -252,7 +255,9 @@ svn_fs_fs__get_rep_reference(representat { *rep = apr_pcalloc(pool, sizeof(**rep)); svn_fs_fs__id_txn_reset(&(*rep)->txn_id); - (*rep)->sha1_checksum = svn_checksum_dup(checksum, pool); + memcpy((*rep)->sha1_digest, checksum->digest, + sizeof((*rep)->sha1_digest)); + (*rep)->has_sha1 = TRUE; (*rep)->revision = svn_sqlite__column_revnum(stmt, 0); (*rep)->offset = svn_sqlite__column_int64(stmt, 1); (*rep)->size = svn_sqlite__column_int64(stmt, 2); @@ -278,20 +283,23 @@ svn_fs_fs__set_rep_reference(svn_fs_t *f fs_fs_data_t *ffd = fs->fsap_data; svn_sqlite__stmt_t *stmt; svn_error_t *err; + svn_checksum_t checksum; + checksum.kind = svn_checksum_sha1; + checksum.digest = rep->sha1_digest; SVN_ERR_ASSERT(ffd->rep_sharing_allowed); if (! ffd->rep_cache_db) SVN_ERR(svn_fs_fs__open_rep_cache(fs, pool)); /* We only allow SHA1 checksums in this table. */ - if (rep->sha1_checksum == NULL) + if (! rep->has_sha1) return svn_error_create(SVN_ERR_BAD_CHECKSUM_KIND, NULL, _("Only SHA1 checksums can be used as keys in the " "rep_cache table.\n")); SVN_ERR(svn_sqlite__get_statement(&stmt, ffd->rep_cache_db, STMT_SET_REP)); SVN_ERR(svn_sqlite__bindf(stmt, "siiii", - svn_checksum_to_cstring(rep->sha1_checksum, pool), + svn_checksum_to_cstring(&checksum, pool), (apr_int64_t) rep->revision, (apr_int64_t) rep->offset, (apr_int64_t) rep->size, @@ -311,8 +319,7 @@ svn_fs_fs__set_rep_reference(svn_fs_t *f should exist. If so, and the value is the same one we were about to write, that's cool -- just do nothing. If, however, the value is *different*, that's a red flag! */ - SVN_ERR(svn_fs_fs__get_rep_reference(&old_rep, fs, rep->sha1_checksum, - pool)); + SVN_ERR(svn_fs_fs__get_rep_reference(&old_rep, fs, &checksum, pool)); if (old_rep) { @@ -329,10 +336,10 @@ svn_fs_fs__set_rep_reference(svn_fs_t *f APR_OFF_T_FMT, SVN_FILESIZE_T_FMT, SVN_FILESIZE_T_FMT, APR_OFF_T_FMT, SVN_FILESIZE_T_FMT, SVN_FILESIZE_T_FMT), - svn_checksum_to_cstring_display(rep->sha1_checksum, pool), - fs->path, old_rep->revision, old_rep->offset, old_rep->size, - old_rep->expanded_size, rep->revision, rep->offset, rep->size, - rep->expanded_size); + svn_checksum_to_cstring_display(&checksum, pool), + fs->path, old_rep->revision, old_rep->offset, + old_rep->size, old_rep->expanded_size, rep->revision, + rep->offset, rep->size, rep->expanded_size); else return SVN_NO_ERROR; } Modified: subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/temp_serializer.c URL: http://svn.apache.org/viewvc/subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/temp_serializer.c?rev=1506622&r1=1506621&r2=1506622&view=diff ============================================================================== --- subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/temp_serializer.c (original) +++ subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/temp_serializer.c Wed Jul 24 16:45:48 2013 @@ -126,43 +126,6 @@ deserialize_svn_string(void *buffer, svn svn_temp_deserializer__resolve(*string, (void **)&(*string)->data); } -/* Utility function to serialize checkum CS within the given serialization - * CONTEXT. - */ -static void -serialize_checksum(svn_temp_serializer__context_t *context, - svn_checksum_t * const *cs) -{ - const svn_checksum_t *checksum = *cs; - if (checksum == NULL) - return; - - svn_temp_serializer__push(context, - (const void * const *)cs, - sizeof(*checksum)); - - /* The digest is arbitrary binary data. - * Thus, we cannot use svn_temp_serializer__add_string. */ - svn_temp_serializer__add_leaf(context, - (const void * const *)&checksum->digest, - svn_checksum_size(checksum)); - - /* return to the caller's nesting level */ - svn_temp_serializer__pop(context); -} - -/* Utility function to deserialize the checksum CS inside the BUFFER. - */ -static void -deserialize_checksum(void *buffer, svn_checksum_t **cs) -{ - svn_temp_deserializer__resolve(buffer, (void **)cs); - if (*cs == NULL) - return; - - svn_temp_deserializer__resolve(*cs, (void **)&(*cs)->digest); -} - /* Utility function to serialize the REPRESENTATION within the given * serialization CONTEXT. */ @@ -175,35 +138,9 @@ serialize_representation(svn_temp_serial return; /* serialize the representation struct itself */ - svn_temp_serializer__push(context, - (const void * const *)representation, - sizeof(*rep)); - - /* serialize sub-structures */ - serialize_checksum(context, &rep->md5_checksum); - serialize_checksum(context, &rep->sha1_checksum); - - /* return to the caller's nesting level */ - svn_temp_serializer__pop(context); -} - -/* Utility function to deserialize the REPRESENTATIONS inside the BUFFER. - */ -static void -deserialize_representation(void *buffer, - representation_t **representation) -{ - representation_t *rep; - - /* fixup the reference to the representation itself */ - svn_temp_deserializer__resolve(buffer, (void **)representation); - rep = *representation; - if (rep == NULL) - return; - - /* fixup of sub-structures */ - deserialize_checksum(rep, &rep->md5_checksum); - deserialize_checksum(rep, &rep->sha1_checksum); + svn_temp_serializer__add_leaf(context, + (const void * const *)representation, + sizeof(*rep)); } /* auxilliary structure representing the content of a directory hash */ @@ -405,8 +342,8 @@ svn_fs_fs__noderev_deserialize(void *buf /* fixup of sub-structures */ svn_fs_fs__id_deserialize(noderev, (svn_fs_id_t **)&noderev->id); svn_fs_fs__id_deserialize(noderev, (svn_fs_id_t **)&noderev->predecessor_id); - deserialize_representation(noderev, &noderev->prop_rep); - deserialize_representation(noderev, &noderev->data_rep); + svn_temp_deserializer__resolve(noderev, (void **)&noderev->prop_rep); + svn_temp_deserializer__resolve(noderev, (void **)&noderev->data_rep); svn_temp_deserializer__resolve(noderev, (void **)&noderev->copyfrom_path); svn_temp_deserializer__resolve(noderev, (void **)&noderev->copyroot_path); Modified: subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/transaction.c URL: http://svn.apache.org/viewvc/subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/transaction.c?rev=1506622&r1=1506621&r2=1506622&view=diff ============================================================================== --- subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/transaction.c (original) +++ subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/transaction.c Wed Jul 24 16:45:48 2013 @@ -54,11 +54,15 @@ static APR_INLINE const char * path_txn_sha1(svn_fs_t *fs, const svn_fs_fs__id_part_t *txn_id, - svn_checksum_t *sha1, + const unsigned char *sha1, apr_pool_t *pool) { + svn_checksum_t checksum; + checksum.digest = sha1; + checksum.kind = svn_checksum_sha1; + return svn_dirent_join(svn_fs_fs__path_txn_dir(fs, txn_id, pool), - svn_checksum_to_cstring(sha1, pool), + svn_checksum_to_cstring(&checksum, pool), pool); } @@ -523,18 +527,17 @@ store_sha1_rep_mapping(svn_fs_t *fs, * its SHA-1 is known, store the rep struct under its SHA1. */ if ( ffd->rep_sharing_allowed && noderev->data_rep - && noderev->data_rep->sha1_checksum) + && noderev->data_rep->has_sha1) { apr_file_t *rep_file; const char *file_name = path_txn_sha1(fs, - svn_fs_fs__id_txn_id(noderev->id), - noderev->data_rep->sha1_checksum, + &noderev->data_rep->txn_id, + noderev->data_rep->sha1_digest, pool); svn_stringbuf_t *rep_string = svn_fs_fs__unparse_representation(noderev->data_rep, ffd->format, (noderev->kind == svn_node_dir), - FALSE, pool); SVN_ERR(svn_io_file_open(&rep_file, file_name, APR_WRITE | APR_CREATE | APR_TRUNCATE @@ -1842,14 +1845,16 @@ get_shared_rep(representation_t **old_re because it is cheepest. */ if (reps_hash) *old_rep = apr_hash_get(reps_hash, - rep->sha1_checksum->digest, + rep->sha1_digest, APR_SHA1_DIGESTSIZE); /* If we haven't found anything yet, try harder and consult our DB. */ if (*old_rep == NULL) { - err = svn_fs_fs__get_rep_reference(old_rep, fs, rep->sha1_checksum, - pool); + svn_checksum_t checksum; + checksum.digest = rep->sha1_digest; + checksum.kind = svn_checksum_sha1; + err = svn_fs_fs__get_rep_reference(old_rep, fs, &checksum, pool); /* ### Other error codes that we shouldn't mask out? */ if (err == SVN_NO_ERROR) { @@ -1887,7 +1892,7 @@ get_shared_rep(representation_t **old_re { svn_node_kind_t kind; const char *file_name - = path_txn_sha1(fs, &rep->txn_id, rep->sha1_checksum, pool); + = path_txn_sha1(fs, &rep->txn_id, rep->sha1_digest, pool); /* in our txn, is there a rep file named with the wanted SHA1? If so, read it and use that rep. @@ -1905,13 +1910,34 @@ get_shared_rep(representation_t **old_re if (*old_rep) { /* Use the old rep for this content. */ - (*old_rep)->md5_checksum = rep->md5_checksum; + memcpy((*old_rep)->md5_digest, rep->md5_digest, sizeof(rep->md5_digest)); (*old_rep)->uniquifier = rep->uniquifier; } return SVN_NO_ERROR; } +/* Copy the hash sum calculation results from MD5_CTX, SHA1_CTX into REP. + * Use POOL for allocations. + */ +static svn_error_t * +digests_final(representation_t *rep, + const svn_checksum_ctx_t *md5_ctx, + const svn_checksum_ctx_t *sha1_ctx, + apr_pool_t *pool) +{ + svn_checksum_t *checksum; + + SVN_ERR(svn_checksum_final(&checksum, md5_ctx, pool)); + memcpy(rep->md5_digest, checksum->digest, svn_checksum_size(checksum)); + SVN_ERR(svn_checksum_final(&checksum, sha1_ctx, pool)); + rep->has_sha1 = checksum != NULL; + if (rep->has_sha1) + memcpy(rep->sha1_digest, checksum->digest, svn_checksum_size(checksum)); + + return SVN_NO_ERROR; +} + /* Close handler for the representation write stream. BATON is a rep_write_baton. Writes out a new node-rev that correctly references the representation we just finished writing. */ @@ -1942,10 +1968,8 @@ rep_write_contents_close(void *baton) rep->revision = SVN_INVALID_REVNUM; /* Finalize the checksum. */ - SVN_ERR(svn_checksum_final(&rep->md5_checksum, b->md5_checksum_ctx, - b->parent_pool)); - SVN_ERR(svn_checksum_final(&rep->sha1_checksum, b->sha1_checksum_ctx, - b->parent_pool)); + SVN_ERR(digests_final(rep, b->md5_checksum_ctx, b->sha1_checksum_ctx, + b->parent_pool)); /* Check and see if we already have a representation somewhere that's identical to the one we just wrote out. */ @@ -2193,8 +2217,7 @@ write_hash_rep(representation_t *rep, SVN_ERR(svn_hash_write2(hash, stream, SVN_HASH_TERMINATOR, pool)); /* Store the results. */ - SVN_ERR(svn_checksum_final(&rep->md5_checksum, whb->md5_ctx, pool)); - SVN_ERR(svn_checksum_final(&rep->sha1_checksum, whb->sha1_ctx, pool)); + SVN_ERR(digests_final(rep, whb->md5_ctx, whb->sha1_ctx, pool)); /* Check and see if we already have a representation somewhere that's identical to the one we just wrote out. */ @@ -2301,8 +2324,7 @@ write_hash_delta_rep(representation_t *r SVN_ERR(svn_stream_close(whb->stream)); /* Store the results. */ - SVN_ERR(svn_checksum_final(&rep->md5_checksum, whb->md5_ctx, pool)); - SVN_ERR(svn_checksum_final(&rep->sha1_checksum, whb->sha1_ctx, pool)); + SVN_ERR(digests_final(rep, whb->md5_ctx, whb->sha1_ctx, pool)); /* Check and see if we already have a representation somewhere that's identical to the one we just wrote out. */ @@ -2595,7 +2617,7 @@ write_final_rev(const svn_fs_id_t **new_ APR_ARRAY_PUSH(reps_to_cache, representation_t *) = copy; apr_hash_set(reps_hash, - copy->sha1_checksum->digest, + copy->sha1_digest, APR_SHA1_DIGESTSIZE, copy); } @@ -2603,11 +2625,11 @@ write_final_rev(const svn_fs_id_t **new_ /* don't serialize SHA1 for dirs to disk (waste of space) */ if (noderev->data_rep && noderev->kind == svn_node_dir) - noderev->data_rep->sha1_checksum = NULL; + noderev->data_rep->has_sha1 = FALSE; /* don't serialize SHA1 for props to disk (waste of space) */ if (noderev->prop_rep) - noderev->prop_rep->sha1_checksum = NULL; + noderev->prop_rep->has_sha1 = FALSE; /* Workaround issue #4031: is-fresh-txn-root in revision files. */ noderev->is_fresh_txn_root = FALSE; Modified: subversion/branches/fsfs-improvements/tools/server-side/svn-rep-sharing-stats.c URL: http://svn.apache.org/viewvc/subversion/branches/fsfs-improvements/tools/server-side/svn-rep-sharing-stats.c?rev=1506622&r1=1506621&r2=1506622&view=diff ============================================================================== --- subversion/branches/fsfs-improvements/tools/server-side/svn-rep-sharing-stats.c (original) +++ subversion/branches/fsfs-improvements/tools/server-side/svn-rep-sharing-stats.c Wed Jul 24 16:45:48 2013 @@ -183,7 +183,8 @@ struct key_t /* What we need to know about a rep. */ struct value_t { - svn_checksum_t *sha1_checksum; + svn_checksum_t checksum; + unsigned char sha1_digest[APR_SHA1_DIGESTSIZE]; apr_uint64_t refcount; }; @@ -201,7 +202,7 @@ static svn_error_t *record(apr_hash_t *r * exist or doesn't have the checksum we are after. (The latter case * often corresponds to node_rev->kind == svn_node_dir.) */ - if (records == NULL || rep == NULL || rep->sha1_checksum == NULL) + if (records == NULL || rep == NULL || !rep->has_sha1) return SVN_NO_ERROR; /* Construct the key. @@ -216,17 +217,19 @@ static svn_error_t *record(apr_hash_t *r if ((value = apr_hash_get(records, key, sizeof(*key)))) { /* Paranoia. */ - SVN_ERR_ASSERT(value->sha1_checksum != NULL); - SVN_ERR_ASSERT(svn_checksum_match(value->sha1_checksum, - rep->sha1_checksum)); + SVN_ERR_ASSERT(memcmp(value->sha1_digest, + rep->sha1_digest, + sizeof(value->sha1_digest))); /* Real work. */ value->refcount++; } else { value = apr_palloc(result_pool, sizeof(*value)); - value->sha1_checksum = svn_checksum_dup(rep->sha1_checksum, result_pool); + value->checksum.digest = value->sha1_digest; + value->checksum.kind = svn_checksum_sha1; value->refcount = 1; + memcpy(value->sha1_digest, rep->sha1_digest, sizeof(value->sha1_digest)); } /* Store them. */ @@ -341,7 +344,7 @@ pretty_print(const char *name, SVN_ERR(svn_cmdline_printf(scratch_pool, "%s %" APR_UINT64_T_FMT " %s\n", name, value->refcount, svn_checksum_to_cstring_display( - value->sha1_checksum, + &value->checksum, scratch_pool))); }