beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Work logged] (BEAM-4794) Move Nexmark and SQL to use the new Schema framework
Date Tue, 17 Jul 2018 18:15:00 GMT

     [ https://issues.apache.org/jira/browse/BEAM-4794?focusedWorklogId=124158&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-124158
]

ASF GitHub Bot logged work on BEAM-4794:
----------------------------------------

                Author: ASF GitHub Bot
            Created on: 17/Jul/18 18:14
            Start Date: 17/Jul/18 18:14
    Worklog Time Spent: 10m 
      Work Description: akedin commented on a change in pull request #5956: [BEAM-4794] Move
SQL and SQL Nexmark to the schema framework
URL: https://github.com/apache/beam/pull/5956#discussion_r203127548
 
 

 ##########
 File path: sdks/java/core/src/main/java/org/apache/beam/sdk/transforms/JsonToRow.java
 ##########
 @@ -98,7 +98,7 @@ public void processElement(ProcessContext context) {
                       context.output(jsonToRow(objectMapper(), context.element()));
                     }
                   }))
-          .setCoder(schema.getRowCoder());
+          .setSchema(schema, SerializableFunctions.identity(), SerializableFunctions.identity());
 
 Review comment:
   +1, some shorthand version would be helpful. Maybe default to `identity()` and have a couple
of overloads of `setSchema()` to allow customization? Or wire it up to schema registry and
default to `identity()` there? 
   
   And is `setSchema` the right place to specify these transforms? Isn't it just conflating
 `.apply(ToRow.withSchema())... .apply(ParDo.of(fromRow()))`? My thought is that if the transforms
are non-trivial, then they are probably better be specified as `ParDos`

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Issue Time Tracking
-------------------

    Worklog Id:     (was: 124158)
    Time Spent: 1h 20m  (was: 1h 10m)

> Move Nexmark and SQL to use the new Schema framework
> ----------------------------------------------------
>
>                 Key: BEAM-4794
>                 URL: https://issues.apache.org/jira/browse/BEAM-4794
>             Project: Beam
>          Issue Type: Sub-task
>          Components: dsl-sql
>            Reporter: Reuven Lax
>            Assignee: Reuven Lax
>            Priority: Major
>          Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> This will allow SQL to accept user types. It will also allow the deletion of a lot of
code.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message