spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From zsxw...@apache.org
Subject spark git commit: [SPARK-13652][CORE] Copy ByteBuffer in sendRpcSync as it will be recycled
Date Fri, 04 Mar 2016 06:53:34 GMT
Repository: spark
Updated Branches:
  refs/heads/branch-1.6 b3a512965 -> 51c676e46


[SPARK-13652][CORE] Copy ByteBuffer in sendRpcSync as it will be recycled

## What changes were proposed in this pull request?

`sendRpcSync` should copy the response content because the underlying buffer will be recycled
and reused.

## How was this patch tested?

Jenkins unit tests.

Author: Shixiong Zhu <shixiong@databricks.com>

Closes #11499 from zsxwing/SPARK-13652.

(cherry picked from commit 465c665db1dc65e3b02c584cf7f8d06b24909b0c)
Signed-off-by: Shixiong Zhu <shixiong@databricks.com>


Project: http://git-wip-us.apache.org/repos/asf/spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/51c676e4
Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/51c676e4
Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/51c676e4

Branch: refs/heads/branch-1.6
Commit: 51c676e46c9be28aa9ac37fda45482b38f2eb1d5
Parents: b3a5129
Author: Shixiong Zhu <shixiong@databricks.com>
Authored: Thu Mar 3 22:53:07 2016 -0800
Committer: Shixiong Zhu <shixiong@databricks.com>
Committed: Thu Mar 3 22:53:21 2016 -0800

----------------------------------------------------------------------
 .../org/apache/spark/network/client/RpcResponseCallback.java  | 7 ++++++-
 .../java/org/apache/spark/network/client/TransportClient.java | 6 +++++-
 2 files changed, 11 insertions(+), 2 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/spark/blob/51c676e4/network/common/src/main/java/org/apache/spark/network/client/RpcResponseCallback.java
----------------------------------------------------------------------
diff --git a/network/common/src/main/java/org/apache/spark/network/client/RpcResponseCallback.java
b/network/common/src/main/java/org/apache/spark/network/client/RpcResponseCallback.java
index 47e93f9..6afc63f 100644
--- a/network/common/src/main/java/org/apache/spark/network/client/RpcResponseCallback.java
+++ b/network/common/src/main/java/org/apache/spark/network/client/RpcResponseCallback.java
@@ -24,7 +24,12 @@ import java.nio.ByteBuffer;
  * failure.
  */
 public interface RpcResponseCallback {
-  /** Successful serialized result from server. */
+  /**
+   * Successful serialized result from server.
+   *
+   * After `onSuccess` returns, `response` will be recycled and its content will become invalid.
+   * Please copy the content of `response` if you want to use it after `onSuccess` returns.
+   */
   void onSuccess(ByteBuffer response);
 
   /** Exception either propagated from server or raised on client side. */

http://git-wip-us.apache.org/repos/asf/spark/blob/51c676e4/network/common/src/main/java/org/apache/spark/network/client/TransportClient.java
----------------------------------------------------------------------
diff --git a/network/common/src/main/java/org/apache/spark/network/client/TransportClient.java
b/network/common/src/main/java/org/apache/spark/network/client/TransportClient.java
index c49ca4d..13debf5 100644
--- a/network/common/src/main/java/org/apache/spark/network/client/TransportClient.java
+++ b/network/common/src/main/java/org/apache/spark/network/client/TransportClient.java
@@ -257,7 +257,11 @@ public class TransportClient implements Closeable {
     sendRpc(message, new RpcResponseCallback() {
       @Override
       public void onSuccess(ByteBuffer response) {
-        result.set(response);
+        ByteBuffer copy = ByteBuffer.allocate(response.remaining());
+        copy.put(response);
+        // flip "copy" to make it readable
+        copy.flip();
+        result.set(copy);
       }
 
       @Override


---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org
For additional commands, e-mail: commits-help@spark.apache.org


Mime
View raw message