beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pei He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-1315) FileBasedSink could write tmp files above the output prefix.
Date Thu, 26 Jan 2017 02:43:26 GMT

    [ https://issues.apache.org/jira/browse/BEAM-1315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15839076#comment-15839076
] 

Pei He commented on BEAM-1315:
------------------------------

My main concern is about the directory permission. (Users only have write access under /home/output/)

I think temp files collision is rare, given it is unlikely for users to write with the same
output prefix in two sinks. (if so, the output files will collide). There is also some randomness
from the DATE when we name the temp directory.

> FileBasedSink could write tmp files above the output prefix.
> ------------------------------------------------------------
>
>                 Key: BEAM-1315
>                 URL: https://issues.apache.org/jira/browse/BEAM-1315
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-core
>            Reporter: Pei He
>             Fix For: 0.5.0
>
>
> FileBasedWriteOperation.TemporaryDirectoryBuilder tempDirBuilder =
>     new FileBasedWriteOperation.TemporaryDirectoryBuilder();
> tempDirBuilder.apply("/home/output/"),
> apply() returns "/home/temp-beam-output-2017-01-25_22-14-49"
> And, it was not under the output prefix "/home/output/".
> Examples of Path resolve:
> Paths.get("/home/output").resolve("temp-beam");
> Paths.get("/home/output/").resolve("temp-beam");
> /home/output/temp-beam
> /home/output/temp-beam
> Paths.get("/home/output").resolve("..").resolve("temp-beam");
> Paths.get("/home/output/").resolve("..").resolve("temp-beam");
> /home/output/../temp-beam
> /home/output/../temp-beam
> Paths.get("/home/output").resolveSibling("temp-beam");
> Paths.get("/home/output/").resolveSibling("temp-beam");
> /home/temp-beam
> /home/temp-beam



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message