subversion-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stef...@apache.org
Subject svn commit: r1546928 [1/5] - in /subversion/branches/fsfs-improvements: ./ subversion/libsvn_fs_fs/ subversion/tests/cmdline/ subversion/tests/cmdline/svntest/ subversion/tests/libsvn_fs_fs/ tools/server-side/
Date Mon, 02 Dec 2013 08:48:01 GMT
Author: stefan2
Date: Mon Dec  2 08:48:00 2013
New Revision: 1546928

URL: http://svn.apache.org/r1546928
Log:
On the fsfs-improvements branch:  Merge all changes from the 
log-addressing branch into this one.  There were no conflicts.

This makes fsfs-improvements the canonical branch for the FSFS
format 7 changes and the former sub-branch may be deleted.

For easier review, the following is a summary of the changes.
Please refer to either the in-source comments or the original
commit messages for details.

BRANCH-README contains a feature overview.  The structure and
structure-indexes files contain a detailed description of the
new addressing concepts and their external representation.

* BRANCH-README
  (): Replace the old description with that for FSFS f7.

* subversion/libsvn_fs_fs/structure
  (): Add format 7 to format list and add index files as new
      elements.  Describe the concept of addressing modes,
      "item index" and "item type" in detail.  Explain the
      intention behind the new pack code.

* subversion/libsvn_fs_fs/structure-indexes
  (): New file.
      Detailed description of the two-way mapping between
      logical and physical addresses.  Also describes the
      binary format used to store the index files on disk.

* subversion/libsvn_fs_fs/index.*
  (): New files.
      Add a FSFS-private API for both indexes providing the
      mapping functionality as defined in structure-indexes.

* subversion/libsvn_fs_fs/rev_file.*
  (): New files.
      Provide a FSFS-private API that combines the 3 physical
      files (rev/pack, 2 x index) into a common data structure
      and ensures that they are being accessed consistently wrt.
      pack state.  This is necessary when there is an svnadmin
      pack running in the background while we read the repo.

* subversion/libsvn_fs_fs/id.*
  (): Replace OFFSET with ITEM_INDEX.
  (svn_fs_fs__id_create_root): New utility function enabled by
                               all root nodes having the same
                               item index.

* subversion/libsvn_fs_fs/util.*
  (): Add new path constructors for index files.  Move item
      to offset mapping and rev / pack file open functions
      to their respective new API files.

* subversion/libsvn_fs_fs/low_level.*
  (): Replace OFFSET with ITEM_INDEX.

* subversion/libsvn_fs_fs/fs.h
  (PATH_*): Path constants for the new index files.
  (CONFIG_*): New fsfs.conf configuration options.
  (SVN_FS_FS__FORMAT_NUMBER): Bump format number.
  (*_t): Replace OFFSET with ITEM_INDEX.
  (fs_fs_data_t): Add config settings and caches for the various
                  index elements.

* subversion/libsvn_fs_fs/caching.c
  (svn_fs_fs__initialize_caches): Initialize new caches.

* subversion/libsvn_fs_fs/fs.c
  (initialize_fs_struct): Default to physical addressing.

* subversion/libsvn_fs_fs/fs_fs.c
  (): Handle new addressing format info and config file options.
      Use a different revision 0 template for format 7.

* subversion/libsvn_fs_fs/cached_data.c
  (): Switch file access function to the new rev_file API.
      Follow the OFFSET to ITEM_INDEX rename.
      Implement the "block read" feature and make all data read
      functionality use the block_read() function.
  (dbg_log_access): Produce a more detailed access log in logical
                    addressing mode.

* subversion/libsvn_fs_fs/transaction.c
  (): Provide generic item_index allocator and index writer
      functions that work in both addressing modes.  Also, add
      wrappers streams that calculate the new low-level FVN-1a
      checksums for everything we write to disk.  Invoke all
      of this when writing data to the proto-rev file.  Finally,
      predict the addressing mode of the future revision and
      upgrade transactions from physical to log addressing mode
      upon misprediction or for old txns.

* subversion/libsvn_fs_fs/pack.h
  (svn_fs_fs__order_dir_entries): In mixed addressing mode repos,
                                  the optimal read order depends
                                  on the revision number.

* subversion/libsvn_fs_fs/pack.c
  (): Implement the new reorder-on-pack feature that is a core
      element of f7.  Details of it are described at the top of
      this file.
  (svn_fs_fs__order_dir_entries): Select the optimal order depending
                                  on the addressing mode.

* subversion/libsvn_fs_fs/tree.c
  (fs_dir_optimal_order): Adapt to pack API change.

* subversion/libsvn_fs_fs/verify.c
  (): In log. addressing mode, verify the forward and backward
      indexes are consistent with one another.  Verify that rev /
      pack file size match the index values and that the low-level
      checksums for every item in the rev / pack file match the
      index data as well.

* subversion/libsvn_fs_fs/hotcopy.c
  (): Copy the new index files as well, if the respective revs
      use logical addressing.  Also, clean them up when a shard
      got packed since the last incremental hotcopy.

* subversion/libsvn_fs_fs/recovery.c
  (): Adapt to new rev_file API and ID API changes.

* subversion/libsvn_fs_fs/rep-cache.c
  (): Follow the OFFSET to ITEM_INDEX rename.

* subversion/tests/cmdline/svnadmin_tests.py
  (): Add very limited f7 index read functionality - just enough
      to manipulate the changed paths lists in f7 as well.  Update
      expected output patterns for f7.  Restrict verify_non_utf8_paths
      to pre-f7 repos as manipulating f7 repos is harder and not
      relevant for the code / test coverage here.

* subversion/tests/cmdline/svntest/main.py
  (is_fs_log_addressing): FSFS f7 also uses log addressing.

* subversion/tests/libsvn_fs_fs
  (): svn:ignore test repositories

* subversion/tests/libsvn_fs_fs/fs-fs-pack-test.c
  (): Write correct addressing mode info and check for the right
      files in packed f7 repos.  Add tests TXN format upgrade tests.

* tools/server-side/svn-rep-sharing-stats.c
  (record): Follow the OFFSET to ITEM_INDEX rename.

Added:
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/index.c
      - copied unchanged from r1546844, subversion/branches/log-addressing/subversion/libsvn_fs_fs/index.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/index.h
      - copied unchanged from r1546844, subversion/branches/log-addressing/subversion/libsvn_fs_fs/index.h
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/rev_file.c
      - copied unchanged from r1546844, subversion/branches/log-addressing/subversion/libsvn_fs_fs/rev_file.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/rev_file.h
      - copied unchanged from r1546844, subversion/branches/log-addressing/subversion/libsvn_fs_fs/rev_file.h
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/structure-indexes
      - copied unchanged from r1546844, subversion/branches/log-addressing/subversion/libsvn_fs_fs/structure-indexes
Modified:
    subversion/branches/fsfs-improvements/   (props changed)
    subversion/branches/fsfs-improvements/BRANCH-README
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/cached_data.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/caching.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.h
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs_fs.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/hotcopy.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/id.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/id.h
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/low_level.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/low_level.h
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/pack.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/pack.h
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/recovery.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/rep-cache.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/structure
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/transaction.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/tree.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/util.c
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/util.h
    subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/verify.c
    subversion/branches/fsfs-improvements/subversion/tests/cmdline/svnadmin_tests.py
    subversion/branches/fsfs-improvements/subversion/tests/cmdline/svntest/main.py
    subversion/branches/fsfs-improvements/subversion/tests/libsvn_fs_fs/   (props changed)
    subversion/branches/fsfs-improvements/subversion/tests/libsvn_fs_fs/fs-fs-pack-test.c
    subversion/branches/fsfs-improvements/tools/server-side/svn-rep-sharing-stats.c

Propchange: subversion/branches/fsfs-improvements/
------------------------------------------------------------------------------
  Merged /subversion/branches/log-addressing:r1509279-1511323,1511325-1511345,1511347-1511362,1511365-1515048,1515050-1515087,1515089-1516314,1516316-1516321,1516323-1516664,1516666-1532409,1532411-1533538,1533540-1533542,1533544-1533815,1533817-1533818,1533820-1535667,1535669-1545967,1545969-1546844

Modified: subversion/branches/fsfs-improvements/BRANCH-README
URL: http://svn.apache.org/viewvc/subversion/branches/fsfs-improvements/BRANCH-README?rev=1546928&r1=1546927&r2=1546928&view=diff
==============================================================================
--- subversion/branches/fsfs-improvements/BRANCH-README (original)
+++ subversion/branches/fsfs-improvements/BRANCH-README Mon Dec  2 08:48:00 2013
@@ -1,6 +1,18 @@
-This is an integration branch to review and format 6 compatible changes
-from the fsfs-format7 branch.  After review, the whole branch will get 
-merged into /trunk.
+This is an integration / development branch to copy / implement and
+review the logical addressing feature originally developed on the
+fsfs-format7 branch.  After review and after the fsfs-improvements
+branch got merged into /trunk, this whole branch will get merged
+into /trunk as well.
+
+Features to implement here:
+
+* support for logical addressing
+* switch to format 7; use log addressing after for new shards created
+  after svnadmin upgrade
+* block read / data prefetch for f7 rev / pack file data
+* reorder data upon pack
+* add checksums to index data
+* improve verification to cover all f7 rev / pack file data
 
 More on this here:
 http://svn.haxx.se/dev/archive-2013-06/0755.shtml

Modified: subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/cached_data.c
URL: http://svn.apache.org/viewvc/subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/cached_data.c?rev=1546928&r1=1546927&r2=1546928&view=diff
==============================================================================
--- subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/cached_data.c (original)
+++ subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/cached_data.c Mon Dec  2 08:48:00 2013
@@ -26,10 +26,12 @@
 
 #include "svn_hash.h"
 #include "svn_ctype.h"
+#include "private/svn_io_private.h"
 #include "private/svn_temp_serializer.h"
 
 #include "fs_fs.h"
 #include "id.h"
+#include "index.h"
 #include "low_level.h"
 #include "pack.h"
 #include "util.h"
@@ -39,16 +41,21 @@
 
 #include "svn_private_config.h"
 
+/* forward-declare. See implementation for the docstring */
+static svn_error_t *
+block_read(void **result,
+           svn_fs_t *fs,
+           svn_revnum_t revision,
+           apr_uint64_t item_index,
+           svn_fs_fs__revision_file_t *revision_file,
+           apr_pool_t *result_pool,
+           apr_pool_t *scratch_pool);
+
+
 /* Defined this to enable access logging via dgb__log_access
 #define SVN_FS_FS__LOG_ACCESS
  */
 
-/* Data / item types.
- */
-#define SVN_FS_FS__ITEM_TYPE_ANY_REP    1  /* item is a representation. */
-#define SVN_FS_FS__ITEM_TYPE_NODEREV    2  /* item is a noderev */
-#define SVN_FS_FS__ITEM_TYPE_CHANGES    3  /* item is a changed paths list */
-
 /* When SVN_FS_FS__LOG_ACCESS has been defined, write a line to console
  * showing where REVISION, ITEM_INDEX is located in FS and use ITEM to
  * show details on it's contents if not NULL.  To support format 6 and
@@ -60,7 +67,7 @@
 static svn_error_t *
 dbg_log_access(svn_fs_t *fs,
                svn_revnum_t revision,
-               apr_off_t offset,
+               apr_uint64_t item_index,
                void *item,
                int item_type,
                apr_pool_t *scratch_pool)
@@ -68,14 +75,17 @@ dbg_log_access(svn_fs_t *fs,
   /* no-op if this macro is not defined */
 #ifdef SVN_FS_FS__LOG_ACCESS
   fs_fs_data_t *ffd = fs->fsap_data;
-  static const char *types[] = {"<n/a>", "rep ", "node ", "chgs "};
+  apr_off_t end_offset = 0;
+  svn_fs_fs__p2l_entry_t *entry = NULL;
+  static const char *types[] = {"<n/a>", "frep ", "drep ", "fprop", "dprop",
+                                "node ", "chgs ", "rep  "};
   const char *description = "";
   const char *type = types[item_type];
   const char *pack = "";
-  apr_off_t offset_in_rev = offset;
+  apr_off_t offset;
 
   /* determine rev / pack file offset */
-  SVN_ERR(svn_fs_fs__item_offset(&offset, fs, revision, offset,
+  SVN_ERR(svn_fs_fs__item_offset(&offset, fs, revision, NULL, item_index,
                                  scratch_pool));
 
   /* constructing the pack file description */
@@ -91,13 +101,13 @@ dbg_log_access(svn_fs_t *fs,
         = node->data_rep
         ? apr_psprintf(scratch_pool, " d=%ld/%" APR_UINT64_T_FMT,
                        node->data_rep->revision,
-                       node->data_rep->offset)
+                       node->data_rep->item_index)
         : "";
       const char *prop_rep
         = node->prop_rep
         ? apr_psprintf(scratch_pool, " p=%ld/%" APR_UINT64_T_FMT,
                        node->prop_rep->revision,
-                       node->prop_rep->offset)
+                       node->prop_rep->item_index)
         : "";
       description = apr_psprintf(scratch_pool, "%s   (pc=%d%s%s)",
                                  node->created_path,
@@ -118,7 +128,7 @@ dbg_log_access(svn_fs_t *fs,
         description = apr_psprintf(scratch_pool,
                                    "  DELTA against %ld/%" APR_UINT64_T_FMT,
                                    header->base_revision,
-                                   header->base_offset);
+                                   header->base_item_index);
     }
   else if (item_type == SVN_FS_FS__ITEM_TYPE_CHANGES && item != NULL)
     {
@@ -134,23 +144,53 @@ dbg_log_access(svn_fs_t *fs,
         }
     }
 
-  printf("%5s%10" APR_UINT64_T_HEX_FMT " %s %7ld %7" APR_UINT64_T_FMT \
-          "   %s\n",
-          pack, (apr_uint64_t)(offset), type, revision, offset_in_rev,
-          description);
+  /* some info is only available in format7 repos */
+  if (svn_fs_fs__use_log_addressing(fs, revision))
+    {
+      /* reverse index lookup: get item description in ENTRY */
+      SVN_ERR(svn_fs_fs__p2l_entry_lookup(&entry, fs, revision, offset,
+                                          scratch_pool));
+      if (entry)
+        {
+          /* more details */
+          end_offset = offset + entry->size;
+          type = types[entry->type];
+        }
+
+      /* line output */
+      printf("%5s%4lx:%04lx -%4lx:%04lx %s %7ld %5"APR_UINT64_T_FMT"   %s\n",
+             pack, (long)(offset / ffd->block_size),
+             (long)(offset % ffd->block_size),
+             (long)(end_offset / ffd->block_size),
+             (long)(end_offset % ffd->block_size),
+             type, revision, item_index, description);
+    }
+  else
+    {
+      /* reduced logging for format 6 and earlier */
+      printf("%5s%10" APR_UINT64_T_HEX_FMT " %s %7ld %7" APR_UINT64_T_FMT \
+             "   %s\n",
+             pack, (apr_uint64_t)(offset), type, revision, item_index,
+             description);
+    }
 
 #endif
 
   return SVN_NO_ERROR;
 }
 
-/* Convenience wrapper around svn_io_file_aligned_seek. */
+/* Convenience wrapper around svn_io_file_aligned_seek, taking filesystem
+   FS instead of a block size. */
 static svn_error_t *
-aligned_seek(apr_file_t *file,
+aligned_seek(svn_fs_t *fs,
+             apr_file_t *file,
+             apr_off_t *buffer_start,
              apr_off_t offset,
              apr_pool_t *pool)
 {
-  return svn_error_trace(svn_io_file_aligned_seek(file, 0, NULL, offset,
+  fs_fs_data_t *ffd = fs->fsap_data;
+  return svn_error_trace(svn_io_file_aligned_seek(file, ffd->block_size,
+                                                  buffer_start, offset,
                                                   pool));
 }
 
@@ -158,19 +198,22 @@ aligned_seek(apr_file_t *file,
    the newly opened file in FILE.  Seek to location OFFSET before
    returning.  Perform temporary allocations in POOL. */
 static svn_error_t *
-open_and_seek_revision(apr_file_t **file,
+open_and_seek_revision(svn_fs_fs__revision_file_t **file,
                        svn_fs_t *fs,
                        svn_revnum_t rev,
-                       apr_off_t offset,
+                       apr_uint64_t item,
                        apr_pool_t *pool)
 {
-  apr_file_t *rev_file;
+  svn_fs_fs__revision_file_t *rev_file;
+  apr_off_t offset = -1;
 
   SVN_ERR(svn_fs_fs__ensure_revision_exists(rev, fs, pool));
 
   SVN_ERR(svn_fs_fs__open_pack_or_rev_file(&rev_file, fs, rev, pool));
-  SVN_ERR(svn_fs_fs__item_offset(&offset, fs, rev, offset, pool));
-  SVN_ERR(aligned_seek(rev_file, offset, pool));
+  SVN_ERR(svn_fs_fs__item_offset(&offset, fs, rev_file, rev, NULL, item,
+                                 pool));
+
+  SVN_ERR(aligned_seek(fs, rev_file->file, NULL, offset, pool));
 
   *file = rev_file;
 
@@ -181,21 +224,18 @@ open_and_seek_revision(apr_file_t **file
    to its position and store the newly opened file in FILE.  Perform
    temporary allocations in POOL. */
 static svn_error_t *
-open_and_seek_transaction(apr_file_t **file,
+open_and_seek_transaction(svn_fs_fs__revision_file_t **file,
                           svn_fs_t *fs,
                           representation_t *rep,
                           apr_pool_t *pool)
 {
-  apr_file_t *rev_file;
-
-  SVN_ERR(svn_io_file_open(&rev_file,
-                           svn_fs_fs__path_txn_proto_rev(fs, &rep->txn_id,
-                                                         pool),
-                           APR_READ | APR_BUFFERED, APR_OS_DEFAULT, pool));
+  apr_off_t offset;
 
-  SVN_ERR(aligned_seek(rev_file, rep->offset, pool));
+  SVN_ERR(svn_fs_fs__open_proto_rev_file(file, fs, &rep->txn_id, pool));
 
-  *file = rev_file;
+  SVN_ERR(svn_fs_fs__item_offset(&offset, fs, NULL, SVN_INVALID_REVNUM,
+                                 &rep->txn_id, rep->item_index, pool));
+  SVN_ERR(aligned_seek(fs, (*file)->file, NULL, offset, pool));
 
   return SVN_NO_ERROR;
 }
@@ -204,13 +244,13 @@ open_and_seek_transaction(apr_file_t **f
    the correct file and seek to the correction location.  Store this
    file in *FILE_P.  Perform any allocations in POOL. */
 static svn_error_t *
-open_and_seek_representation(apr_file_t **file_p,
+open_and_seek_representation(svn_fs_fs__revision_file_t **file_p,
                              svn_fs_t *fs,
                              representation_t *rep,
                              apr_pool_t *pool)
 {
   if (! svn_fs_fs__id_txn_used(&rep->txn_id))
-    return open_and_seek_revision(file_p, fs, rep->revision, rep->offset,
+    return open_and_seek_revision(file_p, fs, rep->revision, rep->item_index,
                                   pool);
   else
     return open_and_seek_transaction(file_p, fs, rep, pool);
@@ -238,16 +278,17 @@ get_node_revision_body(node_revision_t *
                        const svn_fs_id_t *id,
                        apr_pool_t *pool)
 {
-  apr_file_t *revision_file;
   svn_error_t *err;
   svn_boolean_t is_cached = FALSE;
   fs_fs_data_t *ffd = fs->fsap_data;
 
   if (svn_fs_fs__id_is_txn(id))
     {
+      apr_file_t *file;
+
       /* This is a transaction node-rev.  Its storage logic is very
          different from that of rev / pack files. */
-      err = svn_io_file_open(&revision_file,
+      err = svn_io_file_open(&file,
                              svn_fs_fs__path_txn_node_rev(fs, id, pool),
                              APR_READ | APR_BUFFERED, APR_OS_DEFAULT, pool);
       if (err)
@@ -262,15 +303,17 @@ get_node_revision_body(node_revision_t *
         }
 
       SVN_ERR(svn_fs_fs__read_noderev(noderev_p,
-                                      svn_stream_from_aprfile2(revision_file,
+                                      svn_stream_from_aprfile2(file,
                                                                FALSE,
                                                                pool),
                                       pool));
     }
   else
     {
+      svn_fs_fs__revision_file_t *revision_file;
+
       /* noderevs in rev / pack files can be cached */
-      const svn_fs_fs__id_part_t *rev_item = svn_fs_fs__id_rev_offset(id);
+      const svn_fs_fs__id_part_t *rev_item = svn_fs_fs__id_rev_item(id);
       pair_cache_key_t key = { 0 };
       key.revision = rev_item->revision;
       key.second = rev_item->number;
@@ -293,21 +336,36 @@ get_node_revision_body(node_revision_t *
                                      rev_item->revision,
                                      rev_item->number,
                                      pool));
-      SVN_ERR(svn_fs_fs__read_noderev(noderev_p,
-                                      svn_stream_from_aprfile2(revision_file,
-                                                                FALSE,
-                                                                pool),
-                                      pool));
 
-      /* Workaround issue #4031: is-fresh-txn-root in revision files. */
-      (*noderev_p)->is_fresh_txn_root = FALSE;
+      if (svn_fs_fs__use_log_addressing(fs, rev_item->revision))
+        {
+          /* block-read will parse the whole block and will also return
+             the one noderev that we need right now. */
+          SVN_ERR(block_read((void **)noderev_p, fs,
+                             rev_item->revision,
+                             rev_item->number,
+                             revision_file,
+                             pool,
+                             pool));
+          SVN_ERR(svn_fs_fs__close_revision_file(revision_file));
+        }
+      else
+        {
+          /* physical addressing mode reading, parsing and caching */
+          SVN_ERR(svn_fs_fs__read_noderev(noderev_p,
+                                          revision_file->stream,
+                                          pool));
+
+          /* Workaround issue #4031: is-fresh-txn-root in revision files. */
+          (*noderev_p)->is_fresh_txn_root = FALSE;
 
-      /* The noderev is not in cache, yet. Add it, if caching has been enabled. */
-      if (ffd->node_revision_cache)
-        SVN_ERR(svn_cache__set(ffd->node_revision_cache,
-                                &key,
-                                *noderev_p,
-                                pool));
+          /* The noderev is not in cache, yet. Add it, if caching has been enabled. */
+          if (ffd->node_revision_cache)
+            SVN_ERR(svn_cache__set(ffd->node_revision_cache,
+                                   &key,
+                                   *noderev_p,
+                                   pool));
+        }
     }
 
   return SVN_NO_ERROR;
@@ -319,7 +377,7 @@ svn_fs_fs__get_node_revision(node_revisi
                              const svn_fs_id_t *id,
                              apr_pool_t *pool)
 {
-  const svn_fs_fs__id_part_t *rev_offset = svn_fs_fs__id_rev_offset(id);
+  const svn_fs_fs__id_part_t *rev_item = svn_fs_fs__id_rev_item(id);
 
   svn_error_t *err = get_node_revision_body(noderev_p, fs, id, pool);
   if (err && err->apr_err == SVN_ERR_FS_CORRUPT)
@@ -331,8 +389,8 @@ svn_fs_fs__get_node_revision(node_revisi
     }
 
   SVN_ERR(dbg_log_access(fs,
-                         rev_offset->revision,
-                         rev_offset->number,
+                         rev_item->revision,
+                         rev_item->number,
                          *noderev_p,
                          SVN_FS_FS__ITEM_TYPE_NODEREV,
                          pool));
@@ -346,7 +404,7 @@ svn_fs_fs__get_node_revision(node_revisi
    temporary variables from POOL. */
 static svn_error_t *
 get_fs_id_at_offset(svn_fs_id_t **id_p,
-                    apr_file_t *rev_file,
+                    svn_fs_fs__revision_file_t *rev_file,
                     svn_fs_t *fs,
                     svn_revnum_t rev,
                     apr_off_t offset,
@@ -354,10 +412,9 @@ get_fs_id_at_offset(svn_fs_id_t **id_p,
 {
   node_revision_t *noderev;
 
-  SVN_ERR(aligned_seek(rev_file, offset, pool));
+  SVN_ERR(aligned_seek(fs, rev_file->file, NULL, offset, pool));
   SVN_ERR(svn_fs_fs__read_noderev(&noderev,
-                                  svn_stream_from_aprfile2(rev_file, TRUE,
-                                                           pool),
+                                  rev_file->stream,
                                   pool));
 
   /* noderev->id is const, get rid of that */
@@ -434,7 +491,7 @@ get_root_changes_offset(apr_off_t *root_
   SVN_ERR(svn_io_file_seek(rev_file, seek_relative, &offset, pool));
 
   trailer->len = trailer->blocksize-1;
-  SVN_ERR(aligned_seek(rev_file, offset - trailer->len, pool));
+  SVN_ERR(aligned_seek(fs, rev_file, NULL, offset - trailer->len, pool));
 
   /* Read in this last block, from which we will identify the last line. */
   SVN_ERR(svn_io_file_read(rev_file, trailer->data, &trailer->len, pool));
@@ -462,30 +519,37 @@ svn_fs_fs__rev_get_root(svn_fs_id_t **ro
                         apr_pool_t *pool)
 {
   fs_fs_data_t *ffd = fs->fsap_data;
+  SVN_ERR(svn_fs_fs__ensure_revision_exists(rev, fs, pool));
 
-  apr_file_t *revision_file;
-  apr_off_t root_offset;
-  svn_fs_id_t *root_id = NULL;
-  svn_boolean_t is_cached;
+  if (svn_fs_fs__use_log_addressing(fs, rev))
+    {
+      *root_id_p = svn_fs_fs__id_create_root(rev, pool);
+    }
+  else
+    {
+      svn_fs_fs__revision_file_t *revision_file;
+      apr_off_t root_offset;
+      svn_fs_id_t *root_id = NULL;
+      svn_boolean_t is_cached;
 
-  SVN_ERR(svn_fs_fs__ensure_revision_exists(rev, fs, pool));
-  SVN_ERR(svn_cache__get((void **) root_id_p, &is_cached,
-                         ffd->rev_root_id_cache, &rev, pool));
-  if (is_cached)
-    return SVN_NO_ERROR;
+      SVN_ERR(svn_cache__get((void **) root_id_p, &is_cached,
+                            ffd->rev_root_id_cache, &rev, pool));
+      if (is_cached)
+        return SVN_NO_ERROR;
 
-  SVN_ERR(svn_fs_fs__open_pack_or_rev_file(&revision_file, fs, rev, pool));
-  SVN_ERR(get_root_changes_offset(&root_offset, NULL, revision_file, fs, rev,
-                                  pool));
+      SVN_ERR(svn_fs_fs__open_pack_or_rev_file(&revision_file, fs, rev, pool));
+      SVN_ERR(get_root_changes_offset(&root_offset, NULL,
+                                      revision_file->file, fs, rev, pool));
 
-  SVN_ERR(get_fs_id_at_offset(&root_id, revision_file, fs, rev,
-                              root_offset, pool));
+      SVN_ERR(get_fs_id_at_offset(&root_id, revision_file, fs, rev,
+                                  root_offset, pool));
 
-  SVN_ERR(svn_io_file_close(revision_file, pool));
+      SVN_ERR(svn_fs_fs__close_revision_file(revision_file));
 
-  SVN_ERR(svn_cache__set(ffd->rev_root_id_cache, &rev, root_id, pool));
+      SVN_ERR(svn_cache__set(ffd->rev_root_id_cache, &rev, root_id, pool));
 
-  *root_id_p = root_id;
+      *root_id_p = root_id;
+    }
 
   return SVN_NO_ERROR;
 }
@@ -495,10 +559,7 @@ svn_fs_fs__rev_get_root(svn_fs_id_t **ro
 typedef struct shared_file_t
 {
   /* The opened file. NULL while file is not open, yet. */
-  apr_file_t *file;
-
-  /* Stream wrapper around FILE. NULL while file is not open, yet. */
-  svn_stream_t *stream;
+  svn_fs_fs__revision_file_t *rfile;
 
   /* file system to open the file in */
   svn_fs_t *fs;
@@ -518,15 +579,15 @@ typedef struct shared_file_t
 typedef struct rep_state_t
 {
                     /* shared lazy-open rev/pack file structure */
-  shared_file_t *file;
+  shared_file_t *sfile;
                     /* The txdelta window cache to use or NULL. */
   svn_cache__t *window_cache;
                     /* Caches un-deltified windows. May be NULL. */
   svn_cache__t *combined_cache;
                     /* revision containing the representation */
   svn_revnum_t revision;
-                    /* representation's offset in REVISION */
-  apr_uint64_t offset;
+                    /* representation's item index in REVISION */
+  apr_uint64_t item_index;
                     /* length of the header at the start of the rep.
                        0 iff this is rep is stored in a container
                        (i.e. does not have a header) */
@@ -541,16 +602,38 @@ typedef struct rep_state_t
   int chunk_index;  /* number of the window to read */
 } rep_state_t;
 
+/* Simple wrapper around svn_fs_fs__get_file_offset to simplify callers. */
+static svn_error_t *
+get_file_offset(apr_off_t *offset,
+                rep_state_t *rs,
+                apr_pool_t *pool)
+{
+  return svn_error_trace(svn_fs_fs__get_file_offset(offset,
+                                                    rs->sfile->rfile->file,
+                                                    pool));
+}
+
+/* Simple wrapper around svn_io_file_aligned_seek to simplify callers. */
+static svn_error_t *
+rs_aligned_seek(rep_state_t *rs,
+                apr_off_t *buffer_start,
+                apr_off_t offset,
+                apr_pool_t *pool)
+{
+  fs_fs_data_t *ffd = rs->sfile->fs->fsap_data;
+  return svn_error_trace(svn_io_file_aligned_seek(rs->sfile->rfile->file,
+                                                  ffd->block_size,
+                                                  buffer_start, offset,
+                                                  pool));
+}
+
 /* Open FILE->FILE and FILE->STREAM if they haven't been opened, yet. */
 static svn_error_t*
 auto_open_shared_file(shared_file_t *file)
 {
-  if (file->file == NULL)
-    {
-      SVN_ERR(svn_fs_fs__open_pack_or_rev_file(&file->file, file->fs,
-                                               file->revision, file->pool));
-      file->stream = svn_stream_from_aprfile2(file->file, TRUE, file->pool);
-    }
+  if (file->rfile == NULL)
+    SVN_ERR(svn_fs_fs__open_pack_or_rev_file(&file->rfile, file->fs,
+                                             file->revision, file->pool));
 
   return SVN_NO_ERROR;
 }
@@ -562,8 +645,9 @@ auto_set_start_offset(rep_state_t *rs, a
 {
   if (rs->start == -1)
     {
-      SVN_ERR(svn_fs_fs__item_offset(&rs->start, rs->file->fs, rs->revision,
-                                     rs->offset, pool));
+      SVN_ERR(svn_fs_fs__item_offset(&rs->start, rs->sfile->fs,
+                                     rs->sfile->rfile, rs->revision, NULL,
+                                     rs->item_index, pool));
       rs->start += rs->header_size;
     }
 
@@ -579,9 +663,9 @@ auto_read_diff_version(rep_state_t *rs, 
   if (rs->ver == -1)
     {
       char buf[4];
-      SVN_ERR(aligned_seek(rs->file->file, rs->start, pool));
-      SVN_ERR(svn_io_file_read_full2(rs->file->file, buf, sizeof(buf),
-                                     NULL, NULL, pool));
+      SVN_ERR(rs_aligned_seek(rs, NULL, rs->start, pool));
+      SVN_ERR(svn_io_file_read_full2(rs->sfile->rfile->file, buf,
+                                     sizeof(buf), NULL, NULL, pool));
 
       /* ### Layering violation */
       if (! ((buf[0] == 'S') && (buf[1] == 'V') && (buf[2] == 'N')))
@@ -620,7 +704,7 @@ create_rep_state_body(rep_state_t **rep_
    * we can re-use the same, already open file object
    */
   svn_boolean_t reuse_shared_file
-    =    shared_file && *shared_file && (*shared_file)->file
+    =    shared_file && *shared_file && (*shared_file)->rfile
       && SVN_IS_VALID_REVNUM((*shared_file)->revision)
       && (*shared_file)->revision < ffd->min_unpacked_rev
       && rep->revision < ffd->min_unpacked_rev
@@ -630,12 +714,12 @@ create_rep_state_body(rep_state_t **rep_
   representation_cache_key_t key;
   key.revision = rep->revision;
   key.is_packed = rep->revision < ffd->min_unpacked_rev;
-  key.offset = rep->offset;
+  key.item_index = rep->item_index;
 
   /* continue constructing RS and RA */
   rs->size = rep->size;
   rs->revision = rep->revision;
-  rs->offset = rep->offset;
+  rs->item_index = rep->item_index;
   rs->window_cache = ffd->txdelta_window_cache;
   rs->combined_cache = ffd->combined_window_cache;
   rs->ver = -1;
@@ -649,7 +733,7 @@ create_rep_state_body(rep_state_t **rep_
   /* initialize the (shared) FILE member in RS */
   if (reuse_shared_file)
     {
-      rs->file = *shared_file;
+      rs->sfile = *shared_file;
     }
   else
     {
@@ -657,7 +741,7 @@ create_rep_state_body(rep_state_t **rep_
       file->revision = rep->revision;
       file->pool = pool;
       file->fs = fs;
-      rs->file = file;
+      rs->sfile = file;
 
       /* remember the current file, if suggested by the caller */
       if (shared_file)
@@ -675,35 +759,39 @@ create_rep_state_body(rep_state_t **rep_
           /* ... we can re-use the same, already open file object.
            * This implies that we don't read from a txn.
            */
-          rs->file = *shared_file;
-          SVN_ERR(auto_open_shared_file(rs->file));
-          SVN_ERR(svn_fs_fs__get_packed_offset(&offset, fs, rep->revision,
-                                               pool));
-          SVN_ERR(aligned_seek((*shared_file)->file, offset + rep->offset,
-                               pool));
+          rs->sfile = *shared_file;
+          SVN_ERR(auto_open_shared_file(rs->sfile));
+          SVN_ERR(svn_fs_fs__item_offset(&offset, fs, rs->sfile->rfile,
+                                         rep->revision, NULL, rep->item_index,
+                                         pool));
+          SVN_ERR(rs_aligned_seek(rs, NULL, offset, pool));
         }
       else
         {
           /* otherwise, create a new file object.  May or may not be
            * an in-txn file.
            */
-          SVN_ERR(open_and_seek_representation(&rs->file->file, fs, rep,
+          SVN_ERR(open_and_seek_representation(&rs->sfile->rfile, fs, rep,
                                                pool));
-          rs->file->stream = svn_stream_from_aprfile2(rs->file->file, TRUE,
-                                                      rs->file->pool);
         }
 
-      SVN_ERR(svn_fs_fs__read_rep_header(&rh, rs->file->stream, pool));
-      SVN_ERR(svn_fs_fs__get_file_offset(&rs->start, rs->file->file, pool));
+      SVN_ERR(svn_fs_fs__read_rep_header(&rh, rs->sfile->rfile->stream, pool));
+      SVN_ERR(get_file_offset(&rs->start, rs, pool));
 
-      /* cache the rep header if appropriate */
+      /* populate the cache if appropriate */
       if (! svn_fs_fs__id_txn_used(&rep->txn_id))
-        if (ffd->rep_header_cache)
-          SVN_ERR(svn_cache__set(ffd->rep_header_cache, &key, rh, pool));
+        {
+          if (svn_fs_fs__use_log_addressing(fs, rep->revision))
+            SVN_ERR(block_read(NULL, fs, rep->revision, rep->item_index,
+                               rs->sfile->rfile, pool, pool));
+          else
+            if (ffd->rep_header_cache)
+              SVN_ERR(svn_cache__set(ffd->rep_header_cache, &key, rh, pool));
+        }
     }
 
   /* finalize */
-  SVN_ERR(dbg_log_access(fs, rep->revision, rep->offset, rh,
+  SVN_ERR(dbg_log_access(fs, rep->revision, rep->item_index, rh,
                          SVN_FS_FS__ITEM_TYPE_ANY_REP, pool));
 
   rs->header_size = rh->header_size;
@@ -769,12 +857,52 @@ svn_fs_fs__check_rep(representation_t *r
                      void **hint,
                      apr_pool_t *pool)
 {
-  rep_state_t *rs;
-  svn_fs_fs__rep_header_t *rep_header;
+  if (svn_fs_fs__use_log_addressing(fs, rep->revision))
+    {
+      svn_error_t *err;
+      apr_off_t offset;
+      svn_fs_fs__p2l_entry_t *entry;
+      svn_boolean_t is_packed;
+
+      svn_fs_fs__revision_file_t rev_file;
+      svn_fs_fs__init_revision_file(&rev_file, fs, rep->revision, pool);
+      SVN_ERR(svn_fs_fs__item_offset(&offset, fs, &rev_file, rep->revision,
+                                     NULL, rep->item_index, pool));
+
+      is_packed = rev_file.is_packed;
+      err = svn_fs_fs__p2l_entry_lookup(&entry, fs, &rev_file, rep->revision,
+                                        offset, pool);
 
-  /* ### Should this be using read_rep_line() directly? */
-  SVN_ERR(create_rep_state(&rs, &rep_header, (shared_file_t**)hint, rep,
-                           fs, pool));
+      /* retry if the packing state has changed */
+      if (is_packed != rev_file.is_packed)
+        {
+          svn_error_clear(err);
+          return svn_error_trace(svn_fs_fs__check_rep(rep, fs, hint, pool));
+        }
+      else
+        {
+          SVN_ERR(err);
+        }
+
+      if (   entry == NULL
+          || entry->type < SVN_FS_FS__ITEM_TYPE_FILE_REP
+          || entry->type > SVN_FS_FS__ITEM_TYPE_DIR_PROPS)
+        return svn_error_createf(SVN_ERR_REPOS_CORRUPTED, NULL,
+                                 _("No representation found at offset %s "
+                                   "for item %" APR_UINT64_T_FMT
+                                   " in revision %ld"),
+                                 apr_off_t_toa(pool, entry->offset),
+                                 rep->item_index, rep->revision);
+    }
+  else
+    {
+      rep_state_t *rs;
+      svn_fs_fs__rep_header_t *rep_header;
+
+      /* ### Should this be using read_rep_line() directly? */
+      SVN_ERR(create_rep_state(&rs, &rep_header, (shared_file_t**)hint,
+                               rep, fs, pool));
+    }
 
   return SVN_NO_ERROR;
 }
@@ -812,7 +940,7 @@ svn_fs_fs__rep_chain_length(int *chain_l
                                     sub_pool));
 
       base_rep.revision = header->base_revision;
-      base_rep.offset = header->base_offset;
+      base_rep.item_index = header->base_item_index;
       base_rep.size = header->base_length;
       svn_fs_fs__id_txn_reset(&base_rep.txn_id);
       is_delta = header->type == svn_fs_fs__rep_delta;
@@ -891,7 +1019,7 @@ get_window_key(window_cache_key_t *key, 
 {
   assert(rs->revision <= APR_UINT32_MAX);
   key->revision = (apr_uint32_t)rs->revision;
-  key->offset = rs->offset;
+  key->item_index = rs->item_index;
   key->chunk_index = rs->chunk_index;
 
   return key;
@@ -1111,7 +1239,7 @@ build_rep_list(apr_array_header_t **list
         }
 
       rep.revision = rep_header->base_revision;
-      rep.offset = rep_header->base_offset;
+      rep.item_index = rep_header->base_item_index;
       rep.size = rep_header->base_length;
       svn_fs_fs__id_txn_reset(&rep.txn_id);
 
@@ -1177,7 +1305,7 @@ read_delta_window(svn_txdelta_window_t *
   apr_off_t end_offset;
   SVN_ERR_ASSERT(rs->chunk_index <= this_chunk);
 
-  SVN_ERR(dbg_log_access(rs->file->fs, rs->revision, rs->offset,
+  SVN_ERR(dbg_log_access(rs->sfile->fs, rs->revision, rs->item_index,
                          NULL, SVN_FS_FS__ITEM_TYPE_ANY_REP, pool));
 
   /* Read the next window.  But first, try to find it in the cache. */
@@ -1186,23 +1314,42 @@ read_delta_window(svn_txdelta_window_t *
     return SVN_NO_ERROR;
 
   /* someone has to actually read the data from file.  Open it */
-  SVN_ERR(auto_open_shared_file(rs->file));
+  SVN_ERR(auto_open_shared_file(rs->sfile));
+
+  /* invoke the 'block-read' feature for non-txn data.
+     However, don't do that if we are in the middle of some representation,
+     because the block is unlikely to contain other data. */
+  if (   rs->chunk_index == 0
+      && SVN_IS_VALID_REVNUM(rs->revision)
+      && svn_fs_fs__use_log_addressing(rs->sfile->fs, rs->revision))
+    {
+      SVN_ERR(block_read(NULL, rs->sfile->fs, rs->revision, rs->item_index,
+                         rs->sfile->rfile, pool, pool));
+
+      /* reading the whole block probably also provided us with the
+         desired txdelta window */
+      SVN_ERR(get_cached_window(nwin, rs, this_chunk, &is_cached, pool));
+      if (is_cached)
+        return SVN_NO_ERROR;
+    }
+
+  /* data is still not cached -> we need to read it.
+     Make sure we have all the necessary info. */
   SVN_ERR(auto_set_start_offset(rs, pool));
   SVN_ERR(auto_read_diff_version(rs, pool));
 
   /* RS->FILE may be shared between RS instances -> make sure we point
    * to the right data. */
   start_offset = rs->start + rs->current;
-  SVN_ERR(aligned_seek(rs->file->file, start_offset, pool));
+  SVN_ERR(rs_aligned_seek(rs, NULL, start_offset, pool));
 
   /* Skip windows to reach the current chunk if we aren't there yet. */
   while (rs->chunk_index < this_chunk)
     {
-      SVN_ERR(svn_txdelta_skip_svndiff_window(rs->file->file, rs->ver,
-                                              pool));
+      SVN_ERR(svn_txdelta_skip_svndiff_window(rs->sfile->rfile->file,
+                                              rs->ver, pool));
       rs->chunk_index++;
-      SVN_ERR(svn_fs_fs__get_file_offset(&start_offset, rs->file->file,
-                                         pool));
+      SVN_ERR(get_file_offset(&start_offset, rs, pool));
       rs->current = start_offset - rs->start;
       if (rs->current >= rs->size)
         return svn_error_create(SVN_ERR_FS_CORRUPT, NULL,
@@ -1212,9 +1359,9 @@ read_delta_window(svn_txdelta_window_t *
     }
 
   /* Actually read the next window. */
-  SVN_ERR(svn_txdelta_read_svndiff_window(nwin, rs->file->stream, rs->ver,
-                                          pool));
-  SVN_ERR(svn_fs_fs__get_file_offset(&end_offset, rs->file->file, pool));
+  SVN_ERR(svn_txdelta_read_svndiff_window(nwin, rs->sfile->rfile->stream,
+                                          rs->ver, pool));
+  SVN_ERR(get_file_offset(&end_offset, rs, pool));
   rs->current = end_offset - rs->start;
   if (rs->current > rs->size)
     return svn_error_create(SVN_ERR_FS_CORRUPT, NULL,
@@ -1238,15 +1385,15 @@ read_plain_window(svn_stringbuf_t **nwin
   
   /* RS->FILE may be shared between RS instances -> make sure we point
    * to the right data. */
-  SVN_ERR(auto_open_shared_file(rs->file));
+  SVN_ERR(auto_open_shared_file(rs->sfile));
   SVN_ERR(auto_set_start_offset(rs, pool));
 
   offset = rs->start + rs->current;
-  SVN_ERR(aligned_seek(rs->file->file, offset, pool));
+  SVN_ERR(rs_aligned_seek(rs, NULL, offset, pool));
 
   /* Read the plain data. */
   *nwin = svn_stringbuf_create_ensure(size, pool);
-  SVN_ERR(svn_io_file_read_full2(rs->file->file, (*nwin)->data, size,
+  SVN_ERR(svn_io_file_read_full2(rs->sfile->rfile->file, (*nwin)->data, size,
                                  NULL, NULL, pool));
   (*nwin)->data[size] = 0;
 
@@ -1399,13 +1546,13 @@ get_contents(struct rep_read_baton *rb,
           if (((apr_off_t) copy_len) > rs->size - rs->current)
             copy_len = (apr_size_t) (rs->size - rs->current);
 
-          SVN_ERR(auto_open_shared_file(rs->file));
+          SVN_ERR(auto_open_shared_file(rs->sfile));
           SVN_ERR(auto_set_start_offset(rs, rb->pool));
 
           offset = rs->start + rs->current;
-          SVN_ERR(aligned_seek(rs->file->file, offset, rb->pool));
-          SVN_ERR(svn_io_file_read_full2(rs->file->file, cur, copy_len,
-                                         NULL, NULL, rb->pool));
+          SVN_ERR(rs_aligned_seek(rs, NULL, offset, rb->pool));
+          SVN_ERR(svn_io_file_read_full2(rs->sfile->rfile->file, cur,
+                                         copy_len, NULL, NULL, rb->pool));
         }
 
       rs->current += copy_len;
@@ -1532,7 +1679,7 @@ svn_fs_fs__get_contents(svn_stream_t **c
       struct rep_read_baton *rb;
 
       fulltext_cache_key.revision = rep->revision;
-      fulltext_cache_key.second = rep->offset;
+      fulltext_cache_key.second = rep->item_index;
       if (ffd->fulltext_cache && SVN_IS_VALID_REVNUM(rep->revision)
           && fulltext_size_is_cachable(ffd, len))
         {
@@ -1607,7 +1754,7 @@ svn_fs_fs__try_process_file_contents(svn
       pair_cache_key_t fulltext_cache_key = { 0 };
 
       fulltext_cache_key.revision = rep->revision;
-      fulltext_cache_key.second = rep->offset;
+      fulltext_cache_key.second = rep->item_index;
       if (ffd->fulltext_cache && SVN_IS_VALID_REVNUM(rep->revision)
           && fulltext_size_is_cachable(ffd, rep->expanded_size))
         {
@@ -1685,7 +1832,7 @@ svn_fs_fs__get_file_delta_stream(svn_txd
       if (rep_header->type == svn_fs_fs__rep_self_delta
           || (rep_header->type == svn_fs_fs__rep_delta
               && rep_header->base_revision == source->data_rep->revision
-              && rep_header->base_offset == source->data_rep->offset))
+              && rep_header->base_item_index == source->data_rep->item_index))
         {
           /* Create the delta read baton. */
           struct delta_read_baton *drb = apr_pcalloc(pool, sizeof(*drb));
@@ -1696,9 +1843,11 @@ svn_fs_fs__get_file_delta_stream(svn_txd
                                                 delta_read_md5_digest, pool);
           return SVN_NO_ERROR;
         }
-      else
-        if (rep_state->file->file)
-          SVN_ERR(svn_io_file_close(rep_state->file->file, pool));
+      else if (rep_state->sfile->rfile)
+        {
+          SVN_ERR(svn_fs_fs__close_revision_file(rep_state->sfile->rfile));
+          rep_state->sfile->rfile = NULL;
+        }
     }
 
   /* Read both fulltexts and construct a delta. */
@@ -1855,7 +2004,7 @@ locate_dir_cache(svn_fs_t *fs,
       if (noderev->data_rep)
         {
           pair_key->revision = noderev->data_rep->revision;
-          pair_key->second = noderev->data_rep->offset;
+          pair_key->second = noderev->data_rep->item_index;
           *key = pair_key;
         }
       else
@@ -1985,7 +2134,7 @@ svn_fs_fs__get_proplist(apr_hash_t **pro
       pair_cache_key_t key = { 0 };
 
       key.revision = rep->revision;
-      key.second = rep->offset;
+      key.second = rep->item_index;
       if (ffd->properties_cache && SVN_IS_VALID_REVNUM(rep->revision))
         {
           svn_boolean_t is_cached;
@@ -2024,43 +2173,536 @@ svn_fs_fs__get_changes(apr_array_header_
                        apr_pool_t *pool)
 {
   apr_off_t changes_offset;
-  apr_file_t *revision_file;
+  svn_fs_fs__revision_file_t *revision_file;
+  svn_boolean_t found;
   fs_fs_data_t *ffd = fs->fsap_data;
 
   /* try cache lookup first */
 
   if (ffd->changes_cache)
+    SVN_ERR(svn_cache__get((void **) changes, &found, ffd->changes_cache,
+                           &rev, pool));
+
+  if (!found)
     {
-      svn_boolean_t found;
-      SVN_ERR(svn_cache__get((void **) changes, &found, ffd->changes_cache,
-                             &rev, pool));
-      if (found)
+      /* read changes from revision file */
+
+      SVN_ERR(svn_fs_fs__ensure_revision_exists(rev, fs, pool));
+      SVN_ERR(svn_fs_fs__open_pack_or_rev_file(&revision_file, fs, rev,
+                                               pool));
+
+      if (svn_fs_fs__use_log_addressing(fs, rev))
+        {
+          /* 'block-read' will also provide us with the desired data */
+          SVN_ERR(block_read((void **)changes, fs,
+                             rev, SVN_FS_FS__ITEM_INDEX_CHANGES,
+                             revision_file, pool, pool));
+        }
+      else
+        {
+          /* physical addressing mode code path */
+          SVN_ERR(get_root_changes_offset(NULL, &changes_offset,
+                                          revision_file->file, fs, rev,
+                                          pool));
+
+          SVN_ERR(aligned_seek(fs, revision_file->file, NULL, changes_offset,
+                               pool));
+          SVN_ERR(svn_fs_fs__read_changes(changes, revision_file->stream,
+                                          pool));
+
+          /* cache for future reference */
+
+          if (ffd->changes_cache)
+            SVN_ERR(svn_cache__set(ffd->changes_cache, &rev, *changes, pool));
+        }
+
+      SVN_ERR(svn_fs_fs__close_revision_file(revision_file));
+    }
+
+  SVN_ERR(dbg_log_access(fs, rev, changes_offset, *changes,
+                         SVN_FS_FS__ITEM_TYPE_CHANGES, pool));
+
+  return SVN_NO_ERROR;
+}
+
+/* Inialize the representation read state RS for the given REP_HEADER and
+ * p2l index ENTRY.  If not NULL, assign FILE and STREAM to RS.
+ * Use POOL for allocations.
+ */
+static svn_error_t *
+init_rep_state(rep_state_t *rs,
+               svn_fs_fs__rep_header_t *rep_header,
+               svn_fs_t *fs,
+               svn_fs_fs__revision_file_t *file,
+               svn_fs_fs__p2l_entry_t* entry,
+               apr_pool_t *pool)
+{
+  fs_fs_data_t *ffd = fs->fsap_data;
+  shared_file_t *shared_file = apr_pcalloc(pool, sizeof(*shared_file));
+
+  /* this function does not apply to representation containers */
+  SVN_ERR_ASSERT(entry->type >= SVN_FS_FS__ITEM_TYPE_FILE_REP
+                 && entry->type <= SVN_FS_FS__ITEM_TYPE_DIR_PROPS);
+  
+  shared_file->rfile = file;
+  shared_file->fs = fs;
+  shared_file->revision = entry->item.revision;
+  shared_file->pool = pool;
+
+  rs->sfile = shared_file;
+  rs->revision = entry->item.revision;
+  rs->item_index = entry->item.number;
+  rs->header_size = rep_header->header_size;
+  rs->start = entry->offset + rs->header_size;
+  rs->current = rep_header->type == svn_fs_fs__rep_plain ? 0 : 4;
+  rs->size = entry->size - rep_header->header_size - 7;
+  rs->ver = 1;
+  rs->chunk_index = 0;
+  rs->window_cache = ffd->txdelta_window_cache;
+  rs->combined_cache = ffd->combined_window_cache;
+
+  return SVN_NO_ERROR;
+}
+
+/* Walk through all windows in the representation addressed by RS in FS
+ * (excluding the delta bases) and put those not already cached into the
+ * window caches.  As a side effect, return the total sum of all expanded
+ * window sizes in *FULLTEXT_LEN.  Use POOL for temporary allocations.
+ */
+static svn_error_t *
+cache_windows(svn_filesize_t *fulltext_len,
+              svn_fs_t *fs,
+              rep_state_t *rs,
+              apr_pool_t *pool)
+{
+  *fulltext_len = 0;
+
+  while (rs->current < rs->size)
+    {
+      svn_txdelta_window_t *window;
+      apr_off_t start_offset = rs->start + rs->current;
+      apr_off_t end_offset;
+
+      /* navigate to & read the current window */
+      SVN_ERR(rs_aligned_seek(rs, NULL, start_offset, pool));
+      SVN_ERR(svn_txdelta_read_svndiff_window(&window,
+                                              rs->sfile->rfile->stream,
+                                              rs->ver, pool));
+
+      /* aggregate expanded window size */
+      *fulltext_len += window->tview_len;
+
+      /* determine on-disk window size */
+      SVN_ERR(get_file_offset(&end_offset, rs, pool));
+      rs->current = end_offset - rs->start;
+      if (rs->current > rs->size)
+        return svn_error_create(SVN_ERR_FS_CORRUPT, NULL,
+                      _("Reading one svndiff window read beyond "
+                                  "the end of the representation"));
+
+      /* cache the window now */
+      SVN_ERR(set_cached_window(window, rs, pool));
+
+      rs->chunk_index++;
+    }
+
+  return SVN_NO_ERROR;
+}
+
+/* Read all txdelta / plain windows following REP_HEADER in FS as described
+ * by ENTRY.  Read the data from the already open FILE and the wrapping
+ * STREAM object.  Use POOL for allocations.
+ * If caching is not enabled, this is a no-op.
+ */
+static svn_error_t *
+block_read_windows(svn_fs_fs__rep_header_t *rep_header,
+                   svn_fs_t *fs,
+                   svn_fs_fs__revision_file_t *rev_file,
+                   svn_fs_fs__p2l_entry_t* entry,
+                   apr_pool_t *pool)
+{
+  fs_fs_data_t *ffd = fs->fsap_data;
+  rep_state_t rs = { 0 };
+  apr_off_t offset;
+  window_cache_key_t key = { 0 };
+
+  if (   (rep_header->type != svn_fs_fs__rep_plain
+          && !ffd->txdelta_window_cache)
+      || (rep_header->type == svn_fs_fs__rep_plain
+          && !ffd->combined_window_cache))
+    return SVN_NO_ERROR;
+
+  SVN_ERR(init_rep_state(&rs, rep_header, fs, rev_file, entry, pool));
+  
+  /* RS->FILE may be shared between RS instances -> make sure we point
+   * to the right data. */
+  offset = rs.start + rs.current;
+  if (rep_header->type == svn_fs_fs__rep_plain)
+    {
+      svn_stringbuf_t *plaintext;
+      svn_boolean_t is_cached;
+
+      /* already in cache? */
+      SVN_ERR(svn_cache__has_key(&is_cached, rs.combined_cache,
+                                 get_window_key(&key, &rs), pool));
+      if (is_cached)
         return SVN_NO_ERROR;
+
+      /* for larger reps, the header may have crossed a block boundary.
+       * make sure we still read blocks properly aligned, i.e. don't use
+       * plain seek here. */
+      SVN_ERR(aligned_seek(fs, rev_file->file, NULL, offset, pool));
+
+      plaintext = svn_stringbuf_create_ensure(rs.size, pool);
+      SVN_ERR(svn_io_file_read_full2(rev_file->file, plaintext->data,
+                                     rs.size, &plaintext->len, NULL, pool));
+      plaintext->data[plaintext->len] = 0;
+      rs.current += rs.size;
+
+      SVN_ERR(set_cached_combined_window(plaintext, &rs, pool));
+    }
+  else
+    {
+      svn_filesize_t fulltext_len;
+      SVN_ERR(cache_windows(&fulltext_len, fs, &rs, pool));
     }
 
-  /* read changes from revision file */
+  return SVN_NO_ERROR;
+}
 
-  SVN_ERR(svn_fs_fs__ensure_revision_exists(rev, fs, pool));
-  SVN_ERR(svn_fs_fs__open_pack_or_rev_file(&revision_file, fs, rev, pool));
+/* Try to get the representation header identified by KEY from FS's cache.
+ * If it has not been cached, read it from the current position in STREAM
+ * and put it into the cache (if caching has been enabled for rep headers).
+ * Return the result in *REP_HEADER.  Use POOL for allocations.
+ */
+static svn_error_t *
+read_rep_header(svn_fs_fs__rep_header_t **rep_header,
+                svn_fs_t *fs,
+                svn_stream_t *stream,
+                representation_cache_key_t *key,
+                apr_pool_t *pool)
+{
+  fs_fs_data_t *ffd = fs->fsap_data;
+  svn_boolean_t is_cached = FALSE;
+  
+  if (ffd->rep_header_cache)
+    {
+      SVN_ERR(svn_cache__get((void**)rep_header, &is_cached,
+                             ffd->rep_header_cache, key, pool));
+      if (is_cached)
+        return SVN_NO_ERROR;
+    }
 
-  SVN_ERR(get_root_changes_offset(NULL, &changes_offset, revision_file, fs,
-                                  rev, pool));
+  SVN_ERR(svn_fs_fs__read_rep_header(rep_header, stream, pool));
 
-  SVN_ERR(aligned_seek(revision_file, changes_offset, pool));
-  SVN_ERR(svn_fs_fs__read_changes(changes,
-                      svn_stream_from_aprfile2(revision_file, TRUE, pool),
-                                  pool));
+  if (ffd->rep_header_cache)
+    SVN_ERR(svn_cache__set(ffd->rep_header_cache, key, *rep_header, pool));
+
+  return SVN_NO_ERROR;
+}
+
+/* Fetch the representation data (header, txdelta / plain windows)
+ * addressed by ENTRY->ITEM in FS and cache it if caches are enabled.
+ * Read the data from the already open FILE and the wrapping
+ * STREAM object.  Use POOL for allocations.
+ */
+static svn_error_t *
+block_read_contents(svn_fs_t *fs,
+                    svn_fs_fs__revision_file_t *rev_file,
+                    svn_fs_fs__p2l_entry_t* entry,
+                    apr_pool_t *pool)
+{
+  representation_cache_key_t header_key = { 0 };
+  svn_fs_fs__rep_header_t *rep_header;
+
+  header_key.revision = (apr_int32_t)entry->item.revision;
+  header_key.is_packed = svn_fs_fs__is_packed_rev(fs, header_key.revision);
+  header_key.item_index = entry->item.number;
+
+  SVN_ERR(read_rep_header(&rep_header, fs, rev_file->stream, &header_key,
+                          pool));
+  SVN_ERR(block_read_windows(rep_header, fs, rev_file, entry, pool));
+
+  return SVN_NO_ERROR;
+}
 
-  SVN_ERR(svn_io_file_close(revision_file, pool));
+/* For the given FILE in FS and wrapping FILE_STREAM return the stream
+ * object in *STREAM that has the lowest performance overhead for reading
+ * and parsing the item addressed by ENTRY.  Use POOL for allocations.
+ */
+static svn_error_t *
+auto_select_stream(svn_stream_t **stream,
+                   svn_fs_t *fs,
+                   svn_fs_fs__revision_file_t *rev_file,
+                   svn_fs_fs__p2l_entry_t* entry,
+                   apr_pool_t *pool)
+{
+  fs_fs_data_t *ffd = fs->fsap_data;
+
+  /* Item parser might be crossing block boundaries? */
+  if (((entry->offset + entry->size + SVN__LINE_CHUNK_SIZE) ^ entry->offset)
+      >= ffd->block_size)
+    {
+      /* Parsing items that cross block boundaries will cause the file
+         buffer to be re-read and misaligned.  So, read the whole block
+         into memory - it must fit anyways since the parsed object will
+         be even larger.
+       */
+      svn_stringbuf_t *text = svn_stringbuf_create_ensure(entry->size, pool);
+      text->len = entry->size;
+      text->data[text->len] = 0;
+      SVN_ERR(svn_io_file_read_full2(rev_file->file, text->data, text->len,
+                                     NULL, NULL, pool));
+      *stream = svn_stream_from_stringbuf(text, pool);
+    }
+  else
+    {
+      *stream = rev_file->stream;
+    }
+
+  return SVN_NO_ERROR;
+}
+
+/* If not already cached or if MUST_READ is set, read the changed paths
+ * list addressed by ENTRY in FS and retúrn it in *CHANGES.  Cache the
+ * result if caching is enabled.  Read the data from the already open
+ * FILE and wrapping FILE_STREAM.  Use POOL for allocations.
+ */
+static svn_error_t *
+block_read_changes(apr_array_header_t **changes,
+                   svn_fs_t *fs,
+                   svn_fs_fs__revision_file_t *rev_file,
+                   svn_fs_fs__p2l_entry_t *entry,
+                   svn_boolean_t must_read,
+                   apr_pool_t *pool)
+{
+  fs_fs_data_t *ffd = fs->fsap_data;
+  svn_stream_t *stream;
+  if (!must_read && !ffd->changes_cache)
+    return SVN_NO_ERROR;
+
+  /* already in cache? */
+  if (!must_read && ffd->changes_cache)
+    {
+      svn_boolean_t is_cached;
+      SVN_ERR(svn_cache__has_key(&is_cached, ffd->changes_cache,
+                                 &entry->item.revision, pool));
+      if (is_cached)
+        return SVN_NO_ERROR;
+    }
+
+  SVN_ERR(auto_select_stream(&stream, fs, rev_file, entry, pool));
+
+  /* read changes from revision file */
+
+  SVN_ERR(svn_fs_fs__read_changes(changes, stream, pool));
 
   /* cache for future reference */
 
   if (ffd->changes_cache)
-    SVN_ERR(svn_cache__set(ffd->changes_cache, &rev, *changes, pool));
+    SVN_ERR(svn_cache__set(ffd->changes_cache, &entry->item.revision,
+                           *changes, pool));
 
-  SVN_ERR(dbg_log_access(fs, rev, changes_offset, *changes,
-                         SVN_FS_FS__ITEM_TYPE_CHANGES, pool));
+  return SVN_NO_ERROR;
+}
+
+/* If not already cached or if MUST_READ is set, read the nod revision
+ * addressed by ENTRY in FS and retúrn it in *NODEREV_P.  Cache the
+ * result if caching is enabled.  Read the data from the already open
+ * FILE and wrapping FILE_STREAM.  Use POOL for allocations.
+ */
+static svn_error_t *
+block_read_noderev(node_revision_t **noderev_p,
+                   svn_fs_t *fs,
+                   svn_fs_fs__revision_file_t *rev_file,
+                   svn_fs_fs__p2l_entry_t *entry,
+                   svn_boolean_t must_read,
+                   apr_pool_t *pool)
+{
+  fs_fs_data_t *ffd = fs->fsap_data;
+  svn_stream_t *stream;
+
+  pair_cache_key_t key = { 0 };
+  key.revision = entry->item.revision;
+  key.second = entry->item.number;
+
+  if (!must_read && !ffd->node_revision_cache)
+    return SVN_NO_ERROR;
+
+  /* already in cache? */
+  if (!must_read && ffd->node_revision_cache)
+    {
+      svn_boolean_t is_cached;
+      SVN_ERR(svn_cache__has_key(&is_cached, ffd->node_revision_cache,
+                                 &key, pool));
+      if (is_cached)
+        return SVN_NO_ERROR;
+    }
+
+  SVN_ERR(auto_select_stream(&stream, fs, rev_file, entry, pool));
+
+  /* read node rev from revision file */
+
+  SVN_ERR(svn_fs_fs__read_noderev(noderev_p, stream, pool));
+
+  /* Workaround issue #4031: is-fresh-txn-root in revision files. */
+  (*noderev_p)->is_fresh_txn_root = FALSE;
+
+  if (ffd->node_revision_cache)
+    SVN_ERR(svn_cache__set(ffd->node_revision_cache, &key, *noderev_p,
+                           pool));
 
   return SVN_NO_ERROR;
 }
 
+/* Read the whole (e.g. 64kB) block containing ITEM_INDEX of REVISION in FS
+ * and put all data into cache.  If necessary and depending on heuristics,
+ * neighboring blocks may also get read.  The data is being read from
+ * already open REVISION_FILE, which must be the correct rev / pack file
+ * w.r.t. REVISION.
+ *
+ * For noderevs and changed path lists, the item fetched can be allocated
+ * RESULT_POOL and returned in *RESULT.  Otherwise, RESULT must be NULL.
+ */
+static svn_error_t *
+block_read(void **result,
+           svn_fs_t *fs,
+           svn_revnum_t revision,
+           apr_uint64_t item_index,
+           svn_fs_fs__revision_file_t *revision_file,
+           apr_pool_t *result_pool,
+           apr_pool_t *scratch_pool)
+{
+  fs_fs_data_t *ffd = fs->fsap_data;
+  apr_off_t offset, wanted_offset = 0;
+  apr_off_t block_start = 0;
+  apr_array_header_t *entries;
+  int run_count = 0;
+  int i;
+  apr_pool_t *iterpool = svn_pool_create(scratch_pool);
+
+  /* don't try this on transaction protorev files */
+  SVN_ERR_ASSERT(SVN_IS_VALID_REVNUM(revision));
+  
+  /* index lookup: find the OFFSET of the item we *must* read plus (in the
+   * "do-while" block) the list of items in the same block. */
+  SVN_ERR(svn_fs_fs__item_offset(&wanted_offset, fs, revision_file,
+                                 revision, NULL, item_index, iterpool));
+
+  offset = wanted_offset;
+
+  /* Heuristics:
+   *
+   * Read this block.  If the last item crosses the block boundary, read
+   * the next block but stop there.  Because cross-boundary items cause
+   * blocks to be read twice, this heuristics will limit this effect to
+   * approx. 50% of blocks, probably less, while providing a sensible
+   * amount of read-ahead.
+   */
+  do
+    {
+      /* fetch list of items in the block surrounding OFFSET */
+      svn_error_t *err
+        = svn_fs_fs__p2l_index_lookup(&entries, fs, revision_file,
+                                      revision, offset, scratch_pool);
+
+      /* if the revision got packed in the meantime and we still need need
+       * to actually read some item, we retry the whole process */
+      if (err &&
+          revision_file->is_packed != svn_fs_fs__is_packed_rev(fs, revision))
+        {
+          if (result && !*result)
+            {
+              SVN_ERR(svn_fs_fs__reopen_revision_file(revision_file, fs, 
+                                                      revision));
+              SVN_ERR(block_read(result, fs, revision, item_index,
+                                  revision_file, result_pool, scratch_pool));
+            }
+
+          return SVN_NO_ERROR;
+        }
+
+      SVN_ERR(err);
+      SVN_ERR(aligned_seek(fs, revision_file->file, &block_start, offset,
+                           iterpool));
+
+      /* read all items from the block */
+      for (i = 0; i < entries->nelts; ++i)
+        {
+          svn_boolean_t is_result;
+          apr_pool_t *pool;
+          svn_fs_fs__p2l_entry_t* entry;
+
+          svn_pool_clear(iterpool);
+
+          /* skip empty sections */
+          entry = &APR_ARRAY_IDX(entries, i, svn_fs_fs__p2l_entry_t);
+          if (entry->type == SVN_FS_FS__ITEM_TYPE_UNUSED)
+            continue;
+
+          /* the item / container we were looking for? */
+          is_result =    result
+                      && entry->offset == wanted_offset
+                      && entry->item.revision == revision
+                      && entry->item.number == item_index;
+
+          /* select the pool that we want the item to be allocated in */
+          pool = is_result ? result_pool : iterpool;
+
+          /* handle all items that start within this block and are relatively
+           * small (i.e. < block size).  Always read the item we need to return.
+           */
+          if (is_result || (   entry->offset >= block_start
+                            && entry->size < ffd->block_size))
+            {
+              void *item = NULL;
+              SVN_ERR(svn_io_file_seek(revision_file->file, SEEK_SET,
+                                       &entry->offset, iterpool));
+              switch (entry->type)
+                {
+                  case SVN_FS_FS__ITEM_TYPE_FILE_REP:
+                  case SVN_FS_FS__ITEM_TYPE_DIR_REP:
+                  case SVN_FS_FS__ITEM_TYPE_FILE_PROPS:
+                  case SVN_FS_FS__ITEM_TYPE_DIR_PROPS:
+                    SVN_ERR(block_read_contents(fs, revision_file, entry,
+                                                pool));
+                    break;
+
+                  case SVN_FS_FS__ITEM_TYPE_NODEREV:
+                    if (ffd->node_revision_cache || is_result)
+                      SVN_ERR(block_read_noderev((node_revision_t **)&item,
+                                                 fs, revision_file,
+                                                 entry, is_result, pool));
+                    break;
+
+                  case SVN_FS_FS__ITEM_TYPE_CHANGES:
+                    SVN_ERR(block_read_changes((apr_array_header_t **)&item,
+                                               fs, revision_file,
+                                               entry, is_result, pool));
+                    break;
+
+                  default:
+                    break;
+                }
+
+              if (is_result)
+                *result = item;
+
+              /* if we crossed a block boundary, read the remainder of
+               * the last block as well */
+              offset = entry->offset + entry->size;
+              if (offset > block_start + ffd->block_size)
+                ++run_count;
+            }
+        }
+
+    }
+  while(run_count++ == 1); /* can only be true once and only if a block
+                            * boundary got crossed */
+
+  /* if the caller requested a result, we must have provided one by now */
+  assert(!result || *result);
+  svn_pool_destroy(iterpool);
+
+  return SVN_NO_ERROR;
+}

Modified: subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/caching.c
URL: http://svn.apache.org/viewvc/subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/caching.c?rev=1546928&r1=1546927&r2=1546928&view=diff
==============================================================================
--- subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/caching.c (original)
+++ subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/caching.c Mon Dec  2 08:48:00 2013
@@ -25,6 +25,7 @@
 #include "id.h"
 #include "dag.h"
 #include "tree.h"
+#include "index.h"
 #include "temp_serializer.h"
 #include "../libsvn_fs/fs-loader.h"
 
@@ -403,7 +404,7 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
    * - Data that can be reconstructed from other elements has low prio
    *   (e.g. fulltexts, directories etc.)
    * - Index data required to find any of the other data has high prio
-   *   (e.g. noderevs)
+   *   (e.g. noderevs, L2P and P2L index pages)
    * - everthing else should use default prio
    */
 
@@ -655,6 +656,69 @@ svn_fs_fs__initialize_caches(svn_fs_t *f
       ffd->combined_window_cache = NULL;
     }
 
+  if (ffd->format >= SVN_FS_FS__MIN_LOG_ADDRESSING_FORMAT)
+    {
+      SVN_ERR(create_cache(&(ffd->l2p_header_cache),
+                           NULL,
+                           membuffer,
+                           0, 0, /* Do not use inprocess cache */
+                           svn_fs_fs__serialize_l2p_header,
+                           svn_fs_fs__deserialize_l2p_header,
+                           sizeof(pair_cache_key_t),
+                           apr_pstrcat(pool, prefix, "L2P_HEADER",
+                                       (char *)NULL),
+                           SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+                           fs,
+                           no_handler,
+                           fs->pool));
+      SVN_ERR(create_cache(&(ffd->l2p_page_cache),
+                           NULL,
+                           membuffer,
+                           0, 0, /* Do not use inprocess cache */
+                           svn_fs_fs__serialize_l2p_page,
+                           svn_fs_fs__deserialize_l2p_page,
+                           sizeof(svn_fs_fs__page_cache_key_t),
+                           apr_pstrcat(pool, prefix, "L2P_PAGE",
+                                       (char *)NULL),
+                           SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+                           fs,
+                           no_handler,
+                           fs->pool));
+      SVN_ERR(create_cache(&(ffd->p2l_header_cache),
+                           NULL,
+                           membuffer,
+                           0, 0, /* Do not use inprocess cache */
+                           svn_fs_fs__serialize_p2l_header,
+                           svn_fs_fs__deserialize_p2l_header,
+                           sizeof(pair_cache_key_t),
+                           apr_pstrcat(pool, prefix, "P2L_HEADER",
+                                       (char *)NULL),
+                           SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+                           fs,
+                           no_handler,
+                           fs->pool));
+      SVN_ERR(create_cache(&(ffd->p2l_page_cache),
+                           NULL,
+                           membuffer,
+                           0, 0, /* Do not use inprocess cache */
+                           svn_fs_fs__serialize_p2l_page,
+                           svn_fs_fs__deserialize_p2l_page,
+                           sizeof(svn_fs_fs__page_cache_key_t),
+                           apr_pstrcat(pool, prefix, "P2L_PAGE",
+                                       (char *)NULL),
+                           SVN_CACHE__MEMBUFFER_HIGH_PRIORITY,
+                           fs,
+                           no_handler,
+                           fs->pool));
+    }
+  else
+    {
+      ffd->l2p_header_cache = NULL;
+      ffd->l2p_page_cache = NULL;
+      ffd->p2l_header_cache = NULL;
+      ffd->p2l_page_cache = NULL;
+    }
+
   return SVN_NO_ERROR;
 }
 

Modified: subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.c
URL: http://svn.apache.org/viewvc/subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.c?rev=1546928&r1=1546927&r2=1546928&view=diff
==============================================================================
--- subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.c (original)
+++ subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.c Mon Dec  2 08:48:00 2013
@@ -221,6 +221,8 @@ static svn_error_t *
 initialize_fs_struct(svn_fs_t *fs)
 {
   fs_fs_data_t *ffd = apr_pcalloc(fs->pool, sizeof(*ffd));
+  ffd->min_log_addressing_rev = SVN_INVALID_REVNUM;
+
   fs->vtable = &fs_vtable;
   fs->fsap_data = ffd;
   return SVN_NO_ERROR;

Modified: subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.h
URL: http://svn.apache.org/viewvc/subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.h?rev=1546928&r1=1546927&r2=1546928&view=diff
==============================================================================
--- subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.h (original)
+++ subversion/branches/fsfs-improvements/subversion/libsvn_fs_fs/fs.h Mon Dec  2 08:48:00 2013
@@ -71,6 +71,10 @@ extern "C" {
 #define PATH_PACKED           "pack"             /* Packed revision data file */
 #define PATH_EXT_PACKED_SHARD ".pack"            /* Extension for packed
                                                     shards */
+#define PATH_EXT_L2P_INDEX    ".l2p"             /* extension of the log-
+                                                    to-phys index */
+#define PATH_EXT_P2L_INDEX    ".p2l"             /* extension of the phys-
+                                                    to-log index */
 /* If you change this, look at tests/svn_test_fs.c(maybe_install_fsfs_conf) */
 #define PATH_CONFIG           "fsfs.conf"        /* Configuration */
 
@@ -84,6 +88,10 @@ extern "C" {
 #define PATH_EXT_PROPS     ".props"        /* Extension for node props */
 #define PATH_EXT_REV       ".rev"          /* Extension of protorev file */
 #define PATH_EXT_REV_LOCK  ".rev-lock"     /* Extension of protorev lock file */
+#define PATH_TXN_ITEM_INDEX "itemidx"      /* File containing the current item
+                                              index number */
+#define PATH_INDEX          "index"        /* name of index files w/o ext */
+
 /* Names of files in legacy FS formats */
 #define PATH_REV           "rev"           /* Proto rev file */
 #define PATH_REV_LOCK      "rev-lock"      /* Proto rev (write) lock file */
@@ -101,11 +109,19 @@ extern "C" {
 #define CONFIG_SECTION_PACKED_REVPROPS   "packed-revprops"
 #define CONFIG_OPTION_REVPROP_PACK_SIZE  "revprop-pack-size"
 #define CONFIG_OPTION_COMPRESS_PACKED_REVPROPS  "compress-packed-revprops"
+#define CONFIG_SECTION_IO                "io"
+#define CONFIG_OPTION_BLOCK_SIZE         "block-size"
+#define CONFIG_OPTION_L2P_PAGE_SIZE      "l2p-page-size"
+#define CONFIG_OPTION_P2L_PAGE_SIZE      "p2l-page-size"
 
 /* The format number of this filesystem.
    This is independent of the repository format number, and
-   independent of any other FS back ends. */
-#define SVN_FS_FS__FORMAT_NUMBER   6
+   independent of any other FS back ends.
+
+   Note: If you bump this, please update the switch statement in
+         svn_fs_fs__create() as well.
+ */
+#define SVN_FS_FS__FORMAT_NUMBER   7
 
 /* The minimum format number that supports svndiff version 1.  */
 #define SVN_FS_FS__MIN_SVNDIFF1_FORMAT 2
@@ -148,6 +164,9 @@ extern "C" {
 /* The minimum format number that supports packed revprops. */
 #define SVN_FS_FS__MIN_PACKED_REVPROP_FORMAT 6
 
+/* The minimum format number that supports packed revprops. */
+#define SVN_FS_FS__MIN_LOG_ADDRESSING_FORMAT 7
+
 /* Minimum format number that will record moves */
 #define SVN_FS_FS__MIN_MOVE_SUPPORT_FORMAT 7
 
@@ -244,8 +263,8 @@ typedef struct representation_cache_key_
   /* Packed or non-packed representation? */
   svn_boolean_t is_packed;
 
-  /* Item offset of the representation */
-  apr_uint64_t offset;
+  /* Item index of the representation */
+  apr_uint64_t item_index;
 } representation_cache_key_t;
 
 /* Key type that identifies a txdelta window. */
@@ -257,8 +276,8 @@ typedef struct window_cache_key_t
   /* Window number within that representation */
   apr_int32_t chunk_index;
 
-  /* Offset of the representation within REVISION */
-  apr_uint64_t offset;
+  /* Item index of the representation */
+  apr_uint64_t item_index;
 } window_cache_key_t;
 
 /* Private (non-shared) FSFS-specific data for each svn_fs_t object.
@@ -267,10 +286,26 @@ typedef struct fs_fs_data_t
 {
   /* The format number of this FS. */
   int format;
+
   /* The maximum number of files to store per directory (for sharded
      layouts) or zero (for linear layouts). */
   int max_files_per_dir;
 
+  /* The first revision that uses logical addressing.  SVN_INVALID_REVNUM
+     if there is no such revision (pre-f7 or non-sharded).  May be a
+     future revision if the current shard started with physical addressing
+     and is not complete, yet. */
+  svn_revnum_t min_log_addressing_rev;
+
+  /* Rev / pack file read granularity. */
+  apr_int64_t block_size;
+
+  /* Capacity in entries of log-to-phys index pages */
+  apr_int64_t l2p_page_size;
+
+  /* Rev / pack file granularity covered by phys-to-log index pages */
+  apr_int64_t p2l_page_size;
+  
   /* The revision that was youngest, last time we checked. */
   svn_revnum_t youngest_rev_cache;
 
@@ -332,7 +367,7 @@ typedef struct fs_fs_data_t
      the key is window_cache_key_t */
   svn_cache__t *combined_window_cache;
 
-  /* Cache for node_revision_t objects; the key is (revision, id offset) */
+  /* Cache for node_revision_t objects; the key is (revision, item_index) */
   svn_cache__t *node_revision_cache;
 
   /* Cache for change lists as APR arrays of change_t * objects; the key
@@ -340,7 +375,7 @@ typedef struct fs_fs_data_t
   svn_cache__t *changes_cache;
 
   /* Cache for svn_fs_fs__rep_header_t objects; the key is a
-     (revision, is_packed, offset) set */
+     (revision, item index) pair */
   svn_cache__t *rep_header_cache;
 
   /* Cache for svn_mergeinfo_t objects; the key is a combination of
@@ -352,6 +387,23 @@ typedef struct fs_fs_data_t
      if the node has mergeinfo, "0" if it doesn't. */
   svn_cache__t *mergeinfo_existence_cache;
 
+  /* Cache for l2p_header_t objects; the key is (revision, is-packed).
+     Will be NULL for pre-format7 repos */
+  svn_cache__t *l2p_header_cache;
+
+  /* Cache for l2p_page_t objects; the key is svn_fs_fs__page_cache_key_t.
+     Will be NULL for pre-format7 repos */
+  svn_cache__t *l2p_page_cache;
+
+  /* Cache for p2l_header_t objects; the key is (revision, is-packed).
+     Will be NULL for pre-format7 repos */
+  svn_cache__t *p2l_header_cache;
+
+  /* Cache for apr_array_header_t objects containing svn_fs_fs__p2l_entry_t
+     elements; the key is svn_fs_fs__page_cache_key_t.
+     Will be NULL for pre-format7 repos */
+  svn_cache__t *p2l_page_cache;
+
   /* TRUE while the we hold a lock on the write lock file. */
   svn_boolean_t has_write_lock;
 
@@ -449,8 +501,8 @@ typedef struct representation_t
   /* Revision where this representation is located. */
   svn_revnum_t revision;
 
-  /* Offset into the revision file where it is located. */
-  svn_filesize_t offset;
+  /* Item index with the the revision. */
+  apr_uint64_t item_index;
 
   /* The size of the representation in bytes as seen in the revision
      file. */



Mime
View raw message