beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kenneth Knowles (JIRA)" <>
Subject [jira] [Commented] (BEAM-3223) PTransform spec should not reuse FunctionSpec
Date Fri, 17 Nov 2017 23:29:00 GMT


Kenneth Knowles commented on BEAM-3223:

Suppose we have {{PTransformSpec}} and {{UdfSpec}} and {{CoderSpec}} that are all a pair of
URN and payload. This is much more type-friendly and is a standard way of avoiding stupid

Now they each occur in one place - {{PTransform}}, {{SdkFunctionSpec}}, and {{Coder}}, respectively.
So we could just inline the fields. That seems much better to me.

message Udf {
  string urn;
  bytes payload;
  string environment_id;

messsage Coder {
  string id;
  string urn;
  bytes payload;
  repeated string component_coder_ids;

message PTransform {
  .. all the existing stuff ...
  string urn;
  bytes payload;

This makes the differences between the three very obvious.

I am only hesitant because the current protos represent a compromise that made all parties
satisfied, but none happy. So bikeshedding is a danger.

> PTransform spec should not reuse FunctionSpec
> ---------------------------------------------
>                 Key: BEAM-3223
>                 URL:
>             Project: Beam
>          Issue Type: Sub-task
>          Components: beam-model
>            Reporter: Henning Rohde
>            Assignee: Henning Rohde
>              Labels: portability
> We should add a new type instead, TransformSpec, say, or just inline a URN and payload.
It's confusing otherwise.

This message was sent by Atlassian JIRA

View raw message