gobblin-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From a...@apache.org
Subject incubator-gobblin git commit: [GOBBLIN-306] Added documentation calling out an issue when configuring Fork Operators
Date Mon, 13 Nov 2017 10:01:48 GMT
Repository: incubator-gobblin
Updated Branches:
  refs/heads/master 368ff92e3 -> e3f9de1a4


[GOBBLIN-306] Added documentation calling out an issue when configuring Fork Operators

Closes #2164 from kadaan/GOBBLIN-306


Project: http://git-wip-us.apache.org/repos/asf/incubator-gobblin/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-gobblin/commit/e3f9de1a
Tree: http://git-wip-us.apache.org/repos/asf/incubator-gobblin/tree/e3f9de1a
Diff: http://git-wip-us.apache.org/repos/asf/incubator-gobblin/diff/e3f9de1a

Branch: refs/heads/master
Commit: e3f9de1a4be541c74057fd5a4ac202b431206080
Parents: 368ff92
Author: Joel Baranick <joel.baranick@ensighten.com>
Authored: Mon Nov 13 02:01:43 2017 -0800
Committer: Abhishek Tiwari <abhishektiwari.btech@gmail.com>
Committed: Mon Nov 13 02:01:43 2017 -0800

----------------------------------------------------------------------
 .../user-guide/Working-with-the-ForkOperator.md | 27 ++++++++++++++++++++
 1 file changed, 27 insertions(+)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-gobblin/blob/e3f9de1a/gobblin-docs/user-guide/Working-with-the-ForkOperator.md
----------------------------------------------------------------------
diff --git a/gobblin-docs/user-guide/Working-with-the-ForkOperator.md b/gobblin-docs/user-guide/Working-with-the-ForkOperator.md
index 5e4c319..40bf854 100644
--- a/gobblin-docs/user-guide/Working-with-the-ForkOperator.md
+++ b/gobblin-docs/user-guide/Working-with-the-ForkOperator.md
@@ -94,6 +94,33 @@ The `ForkOperator` can have many potential use cases and we have seen the
follow
 
 Generally, a common use case of the `ForkOperator` is to route ingested data records so they
get written to different output locations _conditionally_. The `ForkOperator` also finds common
usage for "dual writes" to different sinks potentially in different formats if the job commit
policy `JobCommitPolicy.COMMIT_ON_FULL_SUCCESS` (or `full` in short) or `JobCommitPolicy.COMMIT_SUCCESSFUL_TASKS`
(or `successful` in short) is used, as explained above. 
 
+Troubleshooting
+--------------------
+
+1) When using Forks with jobs defined as Hocon, you may encounter an error like: 
+    ```
+    com.typesafe.config.ConfigException$BugOrBroken: In the map, path 'converter.classes'
occurs as both the parent object of a value and as a value. Because Map has no defined ordering,
this is a broken situation.
+    at com.typesafe.config.impl.PropertiesParser.fromPathMap(PropertiesParser.java:115)
+    at com.typesafe.config.impl.PropertiesParser.fromPathMap(PropertiesParser.java:82)
+    at com.typesafe.config.impl.ConfigImpl.fromAnyRef(ConfigImpl.java:260)
+    at com.typesafe.config.impl.ConfigImpl.fromPathMap(ConfigImpl.java:200)
+    at com.typesafe.config.ConfigFactory.parseMap(ConfigFactory.java:855)
+    at com.typesafe.config.ConfigFactory.parseMap(ConfigFactory.java:866)
+    at gobblin.runtime.embedded.EmbeddedGobblin.getSysConfig(EmbeddedGobblin.java:497)
+    at gobblin.runtime.embedded.EmbeddedGobblin.runAsync(EmbeddedGobblin.java:442)
+    ```
+    This is because in Hocon a key can have only a single type (see: https://github.com/lightbend/config/blob/master/HOCON.md#java-properties-mapping).
+    To solve this, try writing your config like: 
+    
+    ```
+    converter.classes.ROOT_VALUE="..."
+    ...
+    converter.classes.0="..."
+    ...
+    converter.classes.1="..."
+    ```
+
+
 Example
 --------------------
 


Mime
View raw message