bookkeeper-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From eolive...@apache.org
Subject bookkeeper git commit: BOOKKEEPER-1075: BK LedgerMetadata: more memory-efficient parsing of configs
Date Thu, 25 May 2017 12:32:03 GMT
Repository: bookkeeper
Updated Branches:
  refs/heads/master 7b1eec470 -> 64bedc2f9


BOOKKEEPER-1075: BK LedgerMetadata: more memory-efficient parsing of configs

It is the contribution from Alex Yarmula

commit 9d9d7dd26235a9beda4421b7bed750fea1789076
Author: Alex Yarmula <aktwitter.com>
Date: Wed Sep 23 05:57:30 2015 -0700

BK LedgerMetadata: more memory-efficient parsing of configs
Looking at the most prevalent client-side memory allocations, I noticed that we allocate 4KB
every time we open a ledger. This is caused by allocating a 4KB buffer (in TextFormat.toStringBuilder)
to account for the maximum possible Protobufs message, which is unnecessary in our case: we
know the exact size of the metadata ( << 500 B) and don't need to allocate more.
TextFormat.merge(Readable, Message.Builder) is the current method we use. This changes to
use TextFormat.merge(CharSequence, Message.Builder), which avoids the extra 4K allocation
conversion + an extra StringBuilder.

RB_ID=745700

Author: Alex Yarmula <ak@twitter.com>
Author: Sijie Guo <sijie@apache.org>

Reviewers: Enrico Olivelli, Sijie Guo

Closes #162 from sijie/bk_ledger_metadata_efficiency


Project: http://git-wip-us.apache.org/repos/asf/bookkeeper/repo
Commit: http://git-wip-us.apache.org/repos/asf/bookkeeper/commit/64bedc2f
Tree: http://git-wip-us.apache.org/repos/asf/bookkeeper/tree/64bedc2f
Diff: http://git-wip-us.apache.org/repos/asf/bookkeeper/diff/64bedc2f

Branch: refs/heads/master
Commit: 64bedc2f92d5bb17783c6ea2a2db7ea29170f264
Parents: 7b1eec4
Author: Alex Yarmula <ak@twitter.com>
Authored: Thu May 25 14:31:57 2017 +0200
Committer: eolivelli <eolivelli@apache.org>
Committed: Thu May 25 14:31:57 2017 +0200

----------------------------------------------------------------------
 .../java/org/apache/bookkeeper/client/LedgerMetadata.java | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/bookkeeper/blob/64bedc2f/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java
----------------------------------------------------------------------
diff --git a/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java
b/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java
index f539186..b752148 100644
--- a/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java
+++ b/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/LedgerMetadata.java
@@ -29,6 +29,7 @@ import org.slf4j.LoggerFactory;
 import java.io.BufferedReader;
 import java.io.IOException;
 import java.io.StringReader;
+import java.nio.CharBuffer;
 import java.util.ArrayList;
 import java.util.Arrays;
 import java.util.Iterator;
@@ -393,8 +394,15 @@ public class LedgerMetadata {
             return parseVersion1Config(lc, reader);
         }
 
+        // remaining size is total minus the length of the version line and '\n'
+        char[] configBuffer = new char[config.length() - (versionLine.length() + 1)];
+        if (configBuffer.length != reader.read(configBuffer, 0, configBuffer.length)) {
+            throw new IOException("Invalid metadata buffer");
+        }
+
         LedgerMetadataFormat.Builder builder = LedgerMetadataFormat.newBuilder();
-        TextFormat.merge(reader, builder);
+
+        TextFormat.merge((CharSequence) CharBuffer.wrap(configBuffer), builder);
         LedgerMetadataFormat data = builder.build();
         lc.writeQuorumSize = data.getQuorumSize();        
         if (data.hasCtime()) {


Mime
View raw message