tinkerpop-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From spmalle...@apache.org
Subject [tinkerpop] 02/02: TINKERPOP-2211 Added method to set RequestOptions using with()
Date Wed, 08 May 2019 18:54:35 GMT
This is an automated email from the ASF dual-hosted git repository.

spmallette pushed a commit to branch TINKERPOP-2211
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git

commit bebd9e72c6a32736d890c0029d6581143ee5e139
Author: Stephen Mallette <spmva@genoprime.com>
AuthorDate: Wed May 8 14:53:39 2019 -0400

    TINKERPOP-2211 Added method to set RequestOptions using with()
    
    For bytecode based traversals
---
 CHANGELOG.asciidoc                                 |  2 +
 docs/src/upgrade/release-3.4.x.asciidoc            | 49 ++++++++++++++++++----
 .../gremlin/process/remote/RemoteConnection.java   | 10 +++++
 .../gremlin/structure/io/gryo/GryoVersion.java     |  4 +-
 .../driver/remote/DriverRemoteConnection.java      | 19 ++++++++-
 .../gremlin/server/GremlinServerIntegrateTest.java | 29 +++++++++++++
 6 files changed, 101 insertions(+), 12 deletions(-)

diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index 05f17a9..fa485a1 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -29,6 +29,8 @@ This release also includes changes from <<release-3-3-7, 3.3.7>>.
 * Changed definition of generic in signature of `from(Traversal)` and `to(Traversal)` steps
to use a wildcard rather than `Vertex`.
 * Fixed problem with connection pool sizing and retry.
 * Changed `:>` in Gremlin Console to submit the client-side timeout on each request.
+* Added option to set per-request settings on a `Traversal` submitted via `Bytecode`.
+* Fixed the Gryo registration for `OptionsStrategy` as it was not serializing state properly.
 
 [[release-3-4-1]]
 === TinkerPop 3.4.1 (Release Date: March 18, 2019)
diff --git a/docs/src/upgrade/release-3.4.x.asciidoc b/docs/src/upgrade/release-3.4.x.asciidoc
index 7f598ea..8afcb8f 100644
--- a/docs/src/upgrade/release-3.4.x.asciidoc
+++ b/docs/src/upgrade/release-3.4.x.asciidoc
@@ -27,13 +27,22 @@ image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/ima
 
 Please see the link:https://github.com/apache/tinkerpop/blob/3.4.2/CHANGELOG.asciidoc#release-3-4-2[changelog]
for a complete list of all the modifications that are part of this release.
 
-== TinkerPop 3.4.1
+=== Upgrading for Users
 
-*Release Date: March 18, 2019*
+==== Per Request Options
 
-Please see the link:https://github.com/apache/tinkerpop/blob/3.4.1/CHANGELOG.asciidoc#release-3-4-1[changelog]
for a complete list of all the modifications that are part of this release.
+In 3.4.0, the notion of `RequestOptions` were added so that users could have an easier way
to configure settings on
+individual requests made through the Java driver. While that change provided a way to set
those configurations for
+script based requests, it didn't include options to make those settings in a `Traversal`
submitted via `Bytecode`. In
+this release those settings become available via `with()` step on the `TraversalSource` as
follows:
 
-=== Upgrading for Users
+[source,java]
+----
+GraphTraversalSource g = traversal().withRemote(conf);
+List<Vertex> vertices = g.with(RemoteConnection.PER_REQUEST_TIMEOUT, 500).V().out("knows").toList()
+----
+
+See: link:https://issues.apache.org/jira/browse/TINKERPOP-2211[TINKERPOP-2211]
 
 ==== Gremlin Console Timeout
 
@@ -46,14 +55,35 @@ As of 3.4.0, the Java Driver API allowed for timeout settings to be more
easily
 was modified for this current version to pass the console timeout for each remote submission
thus yielding more
 consistent and intuitive behavior.
 
-link:https://issues.apache.org/jira/browse/TINKERPOP-2203[TINKERPOP-2203]
+See: link:https://issues.apache.org/jira/browse/TINKERPOP-2203[TINKERPOP-2203]
+
+=== Upgrading for Providers
+
+==== Graph Driver Providers
+
+===== Per Request Options
+
+In GraphBinary serialization, Java `write()` and `writeValue()` from `TypeSerializer<T>`
interface now take a Netty's
+`ByteBuf` instance instead of an `ByteBufAllocator`, that way the same buffer instance gets
reused during the write
+of a message. Additionally, we took the opportunity to remove the unused parameter from `ResponseMessageSerializer`.
+
+See: link:https://issues.apache.org/jira/browse/TINKERPOP-2161[TINKERPOP-2161]
+
+== TinkerPop 3.4.1
+
+*Release Date: March 18, 2019*
+
+Please see the link:https://github.com/apache/tinkerpop/blob/3.4.1/CHANGELOG.asciidoc#release-3-4-1[changelog]
for a complete list of all the modifications that are part of this release.
+
+=== Upgrading for Users
 
 ==== Mix SPARQL and Gremlin
 
 In the initial release of `sparql-gremlin` it was only possible to execute a SPARQL query
and have it translate to
 Gremlin. Therefore, it was only possible to write a query like this:
 
-```text
+[source,text]
+----
 gremlin> g.sparql("SELECT ?name ?age WHERE { ?person v:name ?name . ?person v:age ?age
}")
 ==>[name:marko,age:29]
 ==>[name:vadas,age:27]
@@ -66,11 +96,12 @@ gremlin> g.sparql("SELECT * WHERE { }")
 ==>v[4]
 ==>v[5]
 ==>v[6]
-```
+----
 
 In this release, however, it is now possible to further process that result with Gremlin
steps:
 
-```text
+[source,text]
+----
 gremlin> g.sparql("SELECT ?name ?age WHERE { ?person v:name ?name . ?person v:age ?age
}").select("name")
 ==>marko
 ==>vadas
@@ -79,7 +110,7 @@ gremlin> g.sparql("SELECT ?name ?age WHERE { ?person v:name ?name .
?person v:ag
 gremlin> g.sparql("SELECT * WHERE { }").out("knows").values("name")
 ==>vadas
 ==>josh
-```
+----
 
 See: link:https://issues.apache.org/jira/browse/TINKERPOP-2171[TINKERPOP-2171],
 link:http://tinkerpop.apache.org/docs/3.4.1/reference/#sparql-with-gremlin[Reference Documentation]
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteConnection.java
b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteConnection.java
index 0c124e7..08baa06 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteConnection.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/process/remote/RemoteConnection.java
@@ -41,6 +41,16 @@ public interface RemoteConnection extends AutoCloseable {
     public static final String GREMLIN_REMOTE_CONNECTION_CLASS = GREMLIN_REMOTE + "remoteConnectionClass";
 
     /**
+     * Key for configuring a per-request timeout option for a {@code RemoteConnection} .
+     */
+    public static final String PER_REQUEST_TIMEOUT = GREMLIN_REMOTE + "timeout";
+
+    /**
+     * Key for configuring a the batch size option for a {@code RemoteConnection} .
+     */
+    public static final String PER_REQUEST_BATCH_SIZE = GREMLIN_REMOTE + "batchSize";
+
+    /**
      * Submits {@link Traversal} {@link Bytecode} to a server and returns a promise of a
{@link RemoteTraversal}.
      * The {@link RemoteTraversal} is an abstraction over two types of results that can be
returned as part of the
      * response from the server: the results of the {@link Traversal} itself and the side-effects
that it produced.
diff --git a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/gryo/GryoVersion.java
b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/gryo/GryoVersion.java
index 73bac27..6e55343 100644
--- a/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/gryo/GryoVersion.java
+++ b/gremlin-core/src/main/java/org/apache/tinkerpop/gremlin/structure/io/gryo/GryoVersion.java
@@ -343,7 +343,7 @@ public enum GryoVersion {
             add(GryoTypeReg.of(LazyBarrierStrategy.class, 150));
             add(GryoTypeReg.of(MatchPredicateStrategy.class, 151));
             add(GryoTypeReg.of(OrderLimitStrategy.class, 152));
-            add(GryoTypeReg.of(OptionsStrategy.class, 187));
+            add(GryoTypeReg.of(OptionsStrategy.class, 187, new JavaSerializer()));
             add(GryoTypeReg.of(PathProcessorStrategy.class, 153));
             add(GryoTypeReg.of(PathRetractionStrategy.class, 154));
             add(GryoTypeReg.of(CountStrategy.class, 155));
@@ -579,7 +579,7 @@ public enum GryoVersion {
             add(GryoTypeReg.of(LazyBarrierStrategy.class, 150));
             add(GryoTypeReg.of(MatchPredicateStrategy.class, 151));
             add(GryoTypeReg.of(OrderLimitStrategy.class, 152));
-            add(GryoTypeReg.of(OptionsStrategy.class, 187));
+            add(GryoTypeReg.of(OptionsStrategy.class, 187, new JavaSerializer()));
             add(GryoTypeReg.of(PathProcessorStrategy.class, 153));
             add(GryoTypeReg.of(PathRetractionStrategy.class, 154));
             add(GryoTypeReg.of(CountStrategy.class, 155));
diff --git a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/remote/DriverRemoteConnection.java
b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/remote/DriverRemoteConnection.java
index 4413968..09fad00 100644
--- a/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/remote/DriverRemoteConnection.java
+++ b/gremlin-driver/src/main/java/org/apache/tinkerpop/gremlin/driver/remote/DriverRemoteConnection.java
@@ -21,13 +21,17 @@ package org.apache.tinkerpop.gremlin.driver.remote;
 import org.apache.commons.configuration.Configuration;
 import org.apache.tinkerpop.gremlin.driver.Client;
 import org.apache.tinkerpop.gremlin.driver.Cluster;
+import org.apache.tinkerpop.gremlin.driver.RequestOptions;
 import org.apache.tinkerpop.gremlin.process.remote.RemoteConnection;
 import org.apache.tinkerpop.gremlin.process.remote.RemoteConnectionException;
 import org.apache.tinkerpop.gremlin.process.remote.traversal.RemoteTraversal;
 import org.apache.tinkerpop.gremlin.process.traversal.Bytecode;
+import org.apache.tinkerpop.gremlin.process.traversal.TraversalSource;
 import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
+import org.apache.tinkerpop.gremlin.process.traversal.strategy.decoration.OptionsStrategy;
 import org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils;
 
+import java.util.Iterator;
 import java.util.Optional;
 import java.util.concurrent.CompletableFuture;
 
@@ -210,7 +214,20 @@ public class DriverRemoteConnection implements RemoteConnection {
     @Override
     public <E> CompletableFuture<RemoteTraversal<?, E>> submitAsync(final
Bytecode bytecode) throws RemoteConnectionException {
         try {
-            return client.submitAsync(bytecode).thenApply(rs -> new DriverRemoteTraversal<>(rs,
client, attachElements, conf));
+            final Iterator<OptionsStrategy> itty = IteratorUtils.map(
+                    IteratorUtils.filter(bytecode.getSourceInstructions().iterator(),
+                    s -> s.getOperator().equals(TraversalSource.Symbols.withStrategies)
&& s.getArguments()[0] instanceof OptionsStrategy),
+                    os -> (OptionsStrategy) os.getArguments()[0]);
+            final RequestOptions.Builder builder = RequestOptions.build();
+            while (itty.hasNext()) {
+                final OptionsStrategy optionsStrategy = itty.next();
+                if (optionsStrategy.getOptions().containsKey(PER_REQUEST_TIMEOUT))
+                    builder.timeout((long) optionsStrategy.getOptions().get(PER_REQUEST_TIMEOUT));
+                else if (optionsStrategy.getOptions().containsKey(PER_REQUEST_BATCH_SIZE))
+                    builder.batchSize((int) optionsStrategy.getOptions().get(PER_REQUEST_BATCH_SIZE));
+            }
+
+            return client.submitAsync(bytecode, builder.create()).thenApply(rs -> new
DriverRemoteTraversal<>(rs, client, attachElements, conf));
         } catch (Exception ex) {
             throw new RemoteConnectionException(ex);
         }
diff --git a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
index bfa1704..c0877c1 100644
--- a/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
+++ b/gremlin-server/src/test/java/org/apache/tinkerpop/gremlin/server/GremlinServerIntegrateTest.java
@@ -47,6 +47,7 @@ import org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine;
 import org.apache.tinkerpop.gremlin.groovy.jsr223.GroovyCompilerGremlinPlugin;
 import org.apache.tinkerpop.gremlin.groovy.jsr223.customizer.SimpleSandboxExtension;
 import org.apache.tinkerpop.gremlin.jsr223.ScriptFileGremlinPlugin;
+import org.apache.tinkerpop.gremlin.process.remote.RemoteConnection;
 import org.apache.tinkerpop.gremlin.structure.RemoteGraph;
 import org.apache.tinkerpop.gremlin.process.traversal.Traversal;
 import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversal;
@@ -412,6 +413,34 @@ public class GremlinServerIntegrateTest extends AbstractGremlinServerIntegration
     }
 
     @Test
+    public void shouldTimeOutRemoteTraversalWithPerRequestOption() throws Exception {
+        final GraphTraversalSource g = traversal().withRemote(conf);
+
+        try {
+            // tests sleeping thread
+            g.with(RemoteConnection.PER_REQUEST_TIMEOUT, 500L).inject(1).sideEffect(Lambda.consumer("Thread.sleep(10000)")).iterate();
+            fail("This traversal should have timed out");
+        } catch (Exception ex) {
+            final Throwable t = ex.getCause();
+            assertThat(t, instanceOf(ResponseException.class));
+            assertEquals(ResponseStatusCode.SERVER_ERROR_TIMEOUT, ((ResponseException) t).getResponseStatusCode());
+        }
+
+        // make a graph with a cycle in it to force a long run traversal
+        graphGetter.get().traversal().addV("person").as("p").addE("self").to("p").iterate();
+
+        try {
+            // tests an "unending" traversal
+            g.with(RemoteConnection.PER_REQUEST_TIMEOUT, 500L).V().repeat(__.out()).until(__.outE().count().is(0)).iterate();
+            fail("This traversal should have timed out");
+        } catch (Exception ex) {
+            final Throwable t = ex.getCause();
+            assertThat(t, instanceOf(ResponseException.class));
+            assertEquals(ResponseStatusCode.SERVER_ERROR_TIMEOUT, ((ResponseException) t).getResponseStatusCode());
+        }
+    }
+
+    @Test
     public void shouldProduceProperExceptionOnTimeout() throws Exception {
         final Cluster cluster = TestClientFactory.open();
         final Client client = cluster.connect(name.getMethodName());


Mime
View raw message