beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-292) TextIO.Write.to Empty Files
Date Tue, 07 Jun 2016 04:48:21 GMT

    [ https://issues.apache.org/jira/browse/BEAM-292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317839#comment-15317839
] 

ASF GitHub Bot commented on BEAM-292:
-------------------------------------

GitHub user dhalperi opened a pull request:

    https://github.com/apache/incubator-beam/pull/424

    [BEAM-292] Write: always produce at least 1 WriteT

    Write has a degenerate case wherein, if no elements were in the written PCollection,
    the finalize step will get produced with nothing to finalize. This often prevents
    correct operation, for example when a FileBasedSink produces no files instead of one
    empty file.
    
    Catch and handle this case in Write by opening and closing an empty Writer to
    produce a single WriteT.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dhalperi/incubator-beam write-numshards

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/424.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #424
    
----
commit 2cd79b6c760ed6aeae41a9b69bc628cd72a4ac2d
Author: Dan Halperin <dhalperi@google.com>
Date:   2016-06-07T04:46:37Z

    [BEAM-292] Write: always produce at least 1 WriteT
    
    Write has a degenerate case wherein, if no elements were in the written PCollection,
    the finalize step will get produced with nothing to finalize. This often prevents
    correct operation, for example when a FileBasedSink produces no files instead of one
    empty file.
    
    Fix this in Write by opening and closing an empty Writer to produce a single
    Write.

----


> TextIO.Write.to Empty Files
> ---------------------------
>
>                 Key: BEAM-292
>                 URL: https://issues.apache.org/jira/browse/BEAM-292
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-core
>            Reporter: Jesse Anderson
>
> When a PCollection is empty and is written out with TextIO.Write.to, the output file
is unchanged. This makes it seem like the PCollection was not empty. The output file's contents
should be changed to be empty.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message