beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From jkff <...@git.apache.org>
Subject [GitHub] incubator-beam pull request #1050: Makes FileBasedSink use a temporary direc...
Date Tue, 04 Oct 2016 22:51:22 GMT
GitHub user jkff opened a pull request:

    https://github.com/apache/incubator-beam/pull/1050

    Makes FileBasedSink use a temporary directory

    When writing to `/path/to/foo`, temporary files would be written to `/path/too/foo-temp-$uid`
(or something like that), i.e. as siblings of the final output. That could lead to issues
like http://stackoverflow.com/q/39822859/278042
    
    Now, temporary files are written to a path like: `/path/too/temp-beam-foo-$date/$uid`.
This way, the temporary files won't match the same glob as the final output files (even though
they may still fail to be deleted due to eventual consistency issues).

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/jkff/incubator-beam file-sink-tmp-dir

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/1050.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1050
    
----
commit 1c34acdaf4a0c0697c9646934ac163788133347b
Author: Eugene Kirpichov <kirpichov@google.com>
Date:   2016-10-04T22:23:27Z

    Makes FileBasedSink use a temporary directory
    
    When writing to /path/to/foo, temporary files would be
    written to /path/too/foo-temp-$uid (or something like that),
    i.e. as siblings of the final output. That could lead
    to issues like http://stackoverflow.com/q/39822859/278042
    
    Now, temporary files are written to a path like:
    /path/too/temp-beam-foo-$date/$uid.

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message