beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-3243) multiple anonymous DoFn lead to conflicting names
Date Wed, 29 Nov 2017 21:56:00 GMT

    [ https://issues.apache.org/jira/browse/BEAM-3243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16271603#comment-16271603
] 

ASF GitHub Bot commented on BEAM-3243:
--------------------------------------

jkff commented on a change in pull request #4185: BEAM-3243 better error message when there
are conflicting anonymous names
URL: https://github.com/apache/beam/pull/4185#discussion_r153924198
 
 

 ##########
 File path: sdks/java/core/src/main/java/org/apache/beam/sdk/Pipeline.java
 ##########
 @@ -636,4 +649,49 @@ public void visitPrimitiveTransform(Node node) {
       node.getTransform().validate(options);
     }
   }
+
+  private static class TransformToMessage implements Function<PTransform<?, ?>,
String> {
+    @Override
+    public String apply(final PTransform<?, ?> transform) {
+      final Object representant;
+      if (ParDo.SingleOutput.class.isInstance(transform)) {
+        representant = ParDo.SingleOutput.class.cast(
+                transform).getFn();
+      } else if (ParDo.MultiOutput.class.isInstance(transform)) {
+        representant = ParDo.MultiOutput.class.cast(
+                transform).getFn();
+      } else {
+        representant = transform;
+      }
+      return "    - " + representant;
+    }
+  }
+
+  private static class UnstableNameToMessage implements Function<String, String> {
+    private final Multimap<String, PTransform<?, ?>> instances;
+
+    private UnstableNameToMessage(final Multimap<String, PTransform<?, ?>> instancePerName)
{
+      this.instances = instancePerName;
+    }
+
+    @Override
+    public String apply(final String input) {
+      Collection<PTransform<?, ?>> values = null;
+      String currentName = input;
+      // strip the counter appended to the name,
+      // see uniquifyInternal().
+      // common example: ParDo(Anonymous) becoming ParDo(Anonymous)2
+      while (values == null && !currentName.isEmpty()) {
+        currentName = currentName.substring(0, currentName.length() - 1);
 
 Review comment:
   What if the counter is greater than 10, and takes more than 1 digit?
   
   In general this seems brittle: this is just extracting the original non-unique name of
a transform, right? Can't we pass the offending PTransform's here directly somehow, e.g. at
line 532 key things by the original (non-unique) name and traverse entries that have more
than 1 transform? I think then you can also just remove the variable "unstableNames" because
it's redundant w.r.t. the map.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


> multiple anonymous DoFn lead to conflicting names
> -------------------------------------------------
>
>                 Key: BEAM-3243
>                 URL: https://issues.apache.org/jira/browse/BEAM-3243
>             Project: Beam
>          Issue Type: Task
>          Components: sdk-java-core
>            Reporter: Romain Manni-Bucau
>            Assignee: Romain Manni-Bucau
>             Fix For: 2.3.0
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message