beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Kirpichov (JIRA)" <>
Subject [jira] [Commented] (BEAM-1134) Add support of String in MongoDBIO
Date Mon, 12 Dec 2016 18:38:58 GMT


Eugene Kirpichov commented on BEAM-1134:

I'm against making this part of MongoDbIO, because:

1) Beam programming model already provides a means for converting one thing to another - ParDo.
It is not job of MongoDbIO to convert one thing to another: job of MongoDbIO is to read and
write MongoDb.

2) Taken to the limit, the idea that an IO should also do conversion would lead to every IO
being also capable of converting one thing to another, which would obviously be redundant.

3) If MongoDbIO is capable of converting things, this is more confusing for a user who will
need to choose between using MongoDb's built-in conversion capability and writing their own

4) If MongoDbIO converts things not quite in the way the user would like to, the user will
face a choice between changing MongoDbIO (making it further more complex) and writing a slightly
different ParDo (further fragmenting the set of ways to write a non-Document to MongoDb)

Actually I'm going to soon submit a proposal for a "PTransform style guide" - a guide for
developers of reusable composite PTransforms, and one of the points is specifically to NOT
put into the API of a transform things that can be done by simply composing the transform
with a ParDo.

> Add support of String in MongoDBIO
> ----------------------------------
>                 Key: BEAM-1134
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-extensions
>            Reporter: Jean-Baptiste Onofré
>            Assignee: Jean-Baptiste Onofré
> Currently, the {{MongoDbIO}} is dealing with {{Document}}.
> I will add the support of {{String}} converted to {{Document}}.

This message was sent by Atlassian JIRA

View raw message