beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Bradshaw (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-2676) move BeamSqlRow and BeamSqlRowType to sdk/java/core
Date Mon, 31 Jul 2017 18:52:00 GMT

    [ https://issues.apache.org/jira/browse/BEAM-2676?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16107771#comment-16107771
] 

Robert Bradshaw commented on BEAM-2676:
---------------------------------------

The motivation for this came up because (1) we're currently lacking a Generic tuple-like class
(which would be useful for joins and a Java port for https://github.com/tensorflow/transform
, to name a few) and (2) if we have such a class, SQL should use it rather than do its own
thing if at all possible. 

The BeamRow object itself is not at all SQL centric, however BeamRowType is. The original
proposal was to have BeamRow (I would call it BeamRecord, but don't feel that strongly) be
a generic type in core, a sibling to KV, along with an abstract BeamRowType (which, among
other things, vends the Coder from a list of component coders). Then one would have a concrete
BeamSqlRowType subclass of BeamRowType which is instantiated via java.sql.Types. This is the
most forward compatible state without tying the tuple to SQL. 

(I added an item to mark the entire API as Experimental for now, which I think makes a lot
of sense.)

> move BeamSqlRow and BeamSqlRowType to sdk/java/core
> ---------------------------------------------------
>
>                 Key: BEAM-2676
>                 URL: https://issues.apache.org/jira/browse/BEAM-2676
>             Project: Beam
>          Issue Type: Test
>          Components: dsl-sql
>            Reporter: Xu Mingmin
>            Assignee: Xu Mingmin
>              Labels: dsl_sql_merge
>
> BeamSqlRow/BeamSqlRowType is the fundamental of structured data processing in Beam, like
joins, simple projections/expansions. It's more visible to move them to sdk-java-core.
> It contains two parts:
> 1). remove SQL word in the name,
> BeamSqlRow --> BeamRow
> BeamSqlRowType --> BeamRowType
> 2). move from package {{org.apache.beam.dsls.sql.schema}} to {{org.apache.beam.sdk.sd}}
(sd stands for structure data), in module {{beam-sdks-java-core}}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message