beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Kirpichov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1581) JSON source and sink
Date Fri, 07 Apr 2017 18:58:41 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15961285#comment-15961285
] 

Eugene Kirpichov commented on BEAM-1581:
----------------------------------------

Yeah, JacksonIO sounds like good idea. It also has fewer risk that users will come to rely
on it as "the" way to use JSON from Beam while we introduce more ways to use it with different
APIs.

> JSON source and sink
> --------------------
>
>                 Key: BEAM-1581
>                 URL: https://issues.apache.org/jira/browse/BEAM-1581
>             Project: Beam
>          Issue Type: New Feature
>          Components: sdk-java-extensions
>            Reporter: Aviem Zur
>            Assignee: Aviem Zur
>
> JSON source and sink to read/write JSON files.
> Similarly to {{XmlSource}}/{{XmlSink}}, these be a {{JsonSource}}/{{JonSink}} which are
a {{FileBaseSource}}/{{FileBasedSink}}.
> Consider using methods/code (or refactor these) found in {{AsJsons}} and {{ParseJsons}}
> The {{PCollection}} of objects the user passes to the transform should be embedded in
a valid JSON file
> The most common pattern for this is a large object with an array member which holds all
the data objects and other members for metadata.
> Examples of public JSON APIs: https://www.sitepoint.com/10-example-json-files/
> Another pattern used is a file which is simply a JSON array of objects.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message