tinkerpop-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From spmalle...@apache.org
Subject [tinkerpop] 03/17: TINKERPOP-2601 Documentation for Gherkin for providers
Date Mon, 13 Sep 2021 19:16:13 GMT
This is an automated email from the ASF dual-hosted git repository.

spmallette pushed a commit to branch TINKERPOP-2601
in repository https://gitbox.apache.org/repos/asf/tinkerpop.git

commit 72f9b5d97253692ef728dee252c2571990a20d97
Author: Stephen Mallette <stepmall@amazon.com>
AuthorDate: Fri Aug 13 10:58:42 2021 -0400

    TINKERPOP-2601 Documentation for Gherkin for providers
---
 CHANGELOG.asciidoc                                 |   1 +
 docs/src/dev/developer/for-committers.asciidoc     |  17 ++-
 docs/src/dev/provider/index.asciidoc               | 114 ++++++++++++++++++++-
 docs/src/upgrade/release-3.6.x.asciidoc            |  17 +++
 .../tinkerpop/gremlin/features/StepDefinition.java |   3 -
 .../tinkergraph/TinkerGraphFeatureTest.java        |   2 -
 6 files changed, 147 insertions(+), 7 deletions(-)

diff --git a/CHANGELOG.asciidoc b/CHANGELOG.asciidoc
index 8363264..850b6be 100644
--- a/CHANGELOG.asciidoc
+++ b/CHANGELOG.asciidoc
@@ -31,6 +31,7 @@ limitations under the License.
 * Removed `groovy` and `groovy-json` dependencies from `gremlin-driver` as well as related
`JsonBuilder` serialization support.
 * Replaced log4j usage with logback where builds rely on and packaged distributions now contain
the latter.
 * Prevented metrics computation unless the traversal is in a locked state.
+* Exposed Gherkin tests as part of the provider test suite.
 * Bumped to Apache Hadoop 3.3.1.
 * Bumped to Apache Spark 3.1.2.
 
diff --git a/docs/src/dev/developer/for-committers.asciidoc b/docs/src/dev/developer/for-committers.asciidoc
index d819840..8a3bc67 100644
--- a/docs/src/dev/developer/for-committers.asciidoc
+++ b/docs/src/dev/developer/for-committers.asciidoc
@@ -198,6 +198,7 @@ call out this fact.
 The JIRA issues that track removal of deprecated methods should be periodically evaluated
to determine if it is
 prudent to schedule them into a release.
 
+[[developing-tests]]
 == Developing Tests
 
 TinkerPop has a wide variety of test types that help validate its internal code as well as
external provider code.
@@ -224,9 +225,10 @@ environments without a `GraphProvider` being initialized by a suite.
These types
 `Check` instead. Please see link:https://github.com/apache/tinkerpop/blob/e32a4187e4f25e290aabe14007f9087c48a06521/neo4j-gremlin/src/test/java/org/apache/tinkerpop/gremlin/neo4j/structure/NativeNeo4jStructureCheck.java[NativeNeo4jStructureCheck]
 for an example.
 
+[[gremlin-language-test-cases]]
 === Gremlin Language Test Cases
 
-Test cases for the Gremlin Language currently requires that the newly developed test be added
in three places:
+Test cases for the Gremlin Language currently requires that the newly developed test be added
in two places:
 
 1. As a test written in Java in the `gremlin-test` module within the subpackages of
 `org.apache.tinkerpop.gremlin.process.traversal.step`
@@ -465,6 +467,19 @@ Scenario: g_V_outE_drop
   And the graph should return 0 for count of "g.E()"
 ----
 
+[[gherkin-tags]]
+==== Tags
+
+Scenarios have tags associated with them that help identify subsets of tests so that a test
runner can selectively
+include or ignore certain tests. The tags enable the practical and necessary ability for
providers to ignore tests that
+they simply cannot support. It is important to be aware of the following tags when writing
tests as not including a
+tag when one is necessary will cause provider tests to fail:
+
+* *@MultiMetaProperties* - The scenario uses "The Crew" graph as a dataset or otherwise utilizes
multi-properties
+or meta-properties for creating or querying data.
+* *@RemoteOnly* - The scenario uses some Gremlin syntax that cannot be supported outside
of remote test executions. The
+best example of this sort of test would be one that uses the remote `Lambda` syntax.
+
 == Developing Benchmarks
 
 Benchmarks are a useful tool to track performance between TinkerPop versions and also as
tools to aid development
diff --git a/docs/src/dev/provider/index.asciidoc b/docs/src/dev/provider/index.asciidoc
index 1c928f6..480dee0 100644
--- a/docs/src/dev/provider/index.asciidoc
+++ b/docs/src/dev/provider/index.asciidoc
@@ -617,6 +617,29 @@ image:gremlin-edumacated.png[width=225]
 </dependency>
 ----
 
+Providers currently have two approaches to consider when validating their TinkerPop implementations.
The first approach
+comes from the wholly JVM oriented original test suite which was developed in the early days
of TinkerPop 3.x design
+and development. The second approach is available as of 3.6.0, is Gherkin-based and originates
from the Gremlin
+Language Variant test suite which is language agnostic.
+
+The first approach is more complete and more opinionated as to how an implementation should
behave and in many ways
+helpful in getting an implementation semantically correct from the ground up (i.e. first
getting the `Graph` Structure
+API implemented well by getting the Structure Suite to pass which will almost inevitably
ensure that the most of the
+Gremlin language oriented tests in the Process Suite pass early on). On the other hand, the
fact that this test suite
+is rigorous also can make it harder to implement especially if your graph already exists
and behaves in a certain
+fashion.
+
+The second approach only validates Gremlin semantics which is ultimately what users concern
themselves with as that is
+the method by which they will interact with a provider's `Graph`. This test suite is less
concerned with how a
+TinkerPop implementation does what it does, so long as it succeeds at processing Gremlin
traversals. There is
+significant overlap between this test suite and the aforementioned Process Suite.
+
+At this time, it would be wise for providers to implement both approaches as the goal for
TinkerPop is to move away
+from the rigors of the JVM Structure and Process Suites in favor of Gherkin. Over time, the
Structure and Process
+Suites will be deprecated and removed.
+
+==== JVM Test Suite
+
 The operational semantics of any OLTP or OLAP implementation are validated by `gremlin-test`.
To implement these tests,
 provide test case implementations as shown below, where `XXX` below denotes the name of the
graph implementation (e.g.
 TinkerGraph, Neo4jGraph, HadoopGraph, etc.).
@@ -758,7 +781,7 @@ environment variable, depending on your project layout. Some tests require
this
 creating temporary files. The value is typically set to the project build directory. For
example using the Maven
 SureFire Plugin, this is done via the configuration argLine with `-Dbuild.dir=${project.build.directory}`.
 
-==== Checking resource leak
+===== Checking Resource Leaks
 
 The TinkerPop query engine retrieves data by interfacing with the provider using iterators.
These iterators (depending
 on the provider) may hold up resources in the underlying storage layer and hence, it is critical
to close them after
@@ -772,6 +795,95 @@ is provided with TinkerGraph as `TinkerGraphIterator.java`.
 Assertions for leak detection are enabled by default when running the test suite. They can
be temporarily disabled
 by way of a system property - simply set `-DtestIteratorLeaks=false".
 
+[[gherkin-tests-suite]]
+==== Gherkin Test Suite
+
+The Gherkin Test Suite is a language agnostic set of tests that verify Gremlin semantics.
It provides a unified set of
+tests that validate many TinkerPop components internally. The tests themselves can be found
in `gremlin-tests/features`
+(link:https://github.com/apache/tinkerpop/tree/x.y.z/gremlin-test/features[here]) with their
syntax described in the
+link:https://tinkerpop.apache.org/docs/x.y.z/dev/developer/#gremlin-language-test-cases[TinkerPop
Developer Documentation].
+
+TinkerPop provides some infrastructure, for JVM based graphs, to help make it easier for
providers to implement these
+tests against their implementations. This infrastructure is built on `cucumber-java` which
is a dependency of
+`gremlin-test`. There are two main components to implementing the tests:
+
+1. A `org.apache.tinkerpop.gremlin.features.World` implementation which is a class in `gremlin-test`.
+2. A JUnit test class that will act as the runner for the tests with the appropriate annotations
+
+TIP: It may be helpful to get familiar with link:https://cucumber.io/docs/installation/java/[Cucumber]
before
+proceeding with an implementation.
+
+The `World` implementation provides context to the tests and allows providers to intercept
test events that might be
+important to proper execution specific to their implementations. The most important part
of implementing `World` is
+properly implementing the `GraphTraversalSource getGraphTraversalSource(GraphData)` method
which provides to the test
+the `GraphTraversalSource` to execute the test against.
+
+The JUnit test class is really just the test runner. It is a simple class which must include
some Cucumber annotations.
+The following is just an example as taken from TinkerGraph:
+
+[source,java]
+----
+@RunWith(Cucumber.class)
+@CucumberOptions(
+        tags = "not @RemoteOnly",
+        glue = { "org.apache.tinkerpop.gremlin.features" },
+        features = { "../gremlin-test/features" },
+        plugin = {"pretty", "junit:target/cucumber.xml",
+        objectFactory = GuiceFactory.class})
+----
+
+The `@CucumberOptions` that are used are mostly implementation specific, so it will be up
to the provider to make some
+choices as to what is right for their environment. For TinkerGraph, it needed to ignore Gherkin
tests with the
+`@RemoteOnly` tag (the full list of possible tags can be found link:https://tinkerpop.apache.org/docs/current/dev/developer/#gherkin-tags[here]),
+as will most providers. The "glue" will be the same for all test implementers as it refers
to a package containing
+TinkerPop's test infrastructure in `gremlin-test` (unless of course, a provider needs to
develop their own
+infrastructure for some reason). The "features" is the path to the actual Gherkin test files
that should be made
+available locally. The "plugin" defines a JUnit style output, which happens to be understood
by Maven.
+
+The "objectFactory" is the last component. Cucumber relies on dependency injection to get
a `World` implementation into
+the test infrastructure. Providers may choose from multiple available implementations, but
TinkerPop chose to use
+Guice. To follow this approach include the following module:
+
+[source,xml]
+----
+<dependency>
+    <groupId>com.google.inject</groupId>
+    <artifactId>guice</artifactId>
+    <version>4.2.3</version>
+    <scope>test</scope>
+</dependency>
+----
+
+Following the TinkerGraph implementation, there are two classes to construct:
+
+[source,java]
+----
+public class ServiceModule extends AbstractModule {
+    @Override
+    protected void configure() {
+        bind(World.class).to(TinkerGraphWorld.class);
+    }
+}
+
+public class WorldInjectorSource implements InjectorSource {
+    @Override
+    public Injector getInjector() {
+        return Guice.createInjector(Stage.PRODUCTION, CucumberModules.createScenarioModule(),
new ServiceModule());
+    }
+}
+----
+
+The key here is that the `TinkerGraphWorld` implementation gets bound to `World` in the `ServiceModule`
and there is
+a `WorldInjectorSource` that specifies the `ServiceModule` to Cucumber. As a final step,
the provider's test resources
+needs a `cucumber.properties` file with an entry that specifies the `InjectorSource` so that
Guice can find it. Here
+is the example taken from TinkerGraph where the `WorldInjectorSource` is inner class of `TinkerGraphFeatureTest`
+itself.
+
+[source,text]
+----
+guice.injector-source=org.apache.tinkerpop.gremlin.tinkergraph.TinkerGraphFeatureTest$WorldInjectorSource
+----
+
 === Accessibility via GremlinPlugin
 
 image:gremlin-plugin.png[width=100,float=left] The applications distributed with TinkerPop
do not distribute with
diff --git a/docs/src/upgrade/release-3.6.x.asciidoc b/docs/src/upgrade/release-3.6.x.asciidoc
index 2d0c3df..b8286b3 100644
--- a/docs/src/upgrade/release-3.6.x.asciidoc
+++ b/docs/src/upgrade/release-3.6.x.asciidoc
@@ -77,6 +77,23 @@ See: link:https://issues.apache.org/jira/browse/TINKERPOP-2593[TINKERPOP-2593]
 
 ==== Graph System Providers
 
+===== Gherkin Tests
+
+TinkerPop originally introduced Gherkin-based feature tests when GLVs were first introduced
to help provide a language
+agnostic test capability. The Gherkin tests were a near one-to-one copy of the tests of the
Gremlin Process Test Suite
+which focus on Gremlin semantics. Unfortunately, having both JVM tests and Gherkin tests
meant maintaining two sets
+of tests which were testing identical things.
+
+To simplify the ongoing maintenance of the test suite and to make it even easier to contribute
to the enforcement of
+Gremlin semantics, TinkerPop now provides infrastructure in the `gremlin-test` module to
run the Gherkin-based tests.
+For 3.6.0, the old test suite remains intact and is not deprecated, but providers are encouraged
to implement the
+Gherkin tests as they will include newer tests that may not be in the old test suite and
it would be good to gather
+feedback on the new test suite's usage so that when deprecation and removal of the old suite
comes to pass the
+transition will not carry as much friction.
+
+See: link:https://issues.apache.org/jira/browse/TINKERPOP-2601[TINKERPOP-2601],
+link:https://tinkerpop.apache.org/docs/3.6.0/dev/provider/#gherkin-tests-suite[Provider Documentation]
+
 ===== Filters with Mixed Id Types
 
 The requirement that "ids" passed to `Graph.vertices` and `Graph.edges` all be of a single
type has been removed. This
diff --git a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/features/StepDefinition.java
b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/features/StepDefinition.java
index 3bd136b..b769e21 100644
--- a/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/features/StepDefinition.java
+++ b/gremlin-test/src/main/java/org/apache/tinkerpop/gremlin/features/StepDefinition.java
@@ -187,9 +187,6 @@ public final class StepDefinition {
 
     @Before
     public void beforeEachScenario(final Scenario scenario) throws Exception {
-        if (scenario.getName().equals("g_VX1_2_3_4X_name"))
-            System.out.println("stop!");
-
         world.beforeEachScenario(scenario);
         stringParameters.clear();
         if (traversal != null) {
diff --git a/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/TinkerGraphFeatureTest.java
b/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/TinkerGraphFeatureTest.java
index 478fee1..3cc834d 100644
--- a/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/TinkerGraphFeatureTest.java
+++ b/tinkergraph-gremlin/src/test/java/org/apache/tinkerpop/gremlin/tinkergraph/TinkerGraphFeatureTest.java
@@ -25,7 +25,6 @@ import com.google.inject.Stage;
 import io.cucumber.guice.CucumberModules;
 import io.cucumber.guice.GuiceFactory;
 import io.cucumber.guice.InjectorSource;
-import io.cucumber.java.Scenario;
 import io.cucumber.junit.Cucumber;
 import io.cucumber.junit.CucumberOptions;
 import org.apache.commons.configuration2.BaseConfiguration;
@@ -34,7 +33,6 @@ import org.apache.tinkerpop.gremlin.features.World;
 import org.apache.tinkerpop.gremlin.process.traversal.dsl.graph.GraphTraversalSource;
 import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerFactory;
 import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph;
-import org.junit.AssumptionViolatedException;
 import org.junit.runner.RunWith;
 
 import static org.apache.tinkerpop.gremlin.LoadGraphWith.GraphData;

Mime
View raw message