flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nico Kruber (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (FLINK-6380) BlobService concurrency issues between delete and put/get methods
Date Fri, 06 Oct 2017 09:49:00 GMT

     [ https://issues.apache.org/jira/browse/FLINK-6380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Nico Kruber closed FLINK-6380.
       Resolution: Fixed
    Fix Version/s: 1.4.0

This should be fixed by FLINK-7068. Please re-open if it occurs again.

> BlobService concurrency issues between delete and put/get methods
> -----------------------------------------------------------------
>                 Key: FLINK-6380
>                 URL: https://issues.apache.org/jira/browse/FLINK-6380
>             Project: Flink
>          Issue Type: Bug
>          Components: Network
>    Affects Versions: 1.3.0
>            Reporter: Nico Kruber
>            Assignee: Nico Kruber
>             Fix For: 1.4.0
> {{BlobCache#deleteAll(JobID)}} deletes the job directory which is only created at the
start of {{BlobCache#getURL(BlobKey)}} which then relies on the directory being present.
> This is not restricted to the {{BlobCache}}, though, but also affects the {{BlobServer}}
in two ways:
> 1) its own local storage and
> 2) its backing {{BlobStore}}
> For the latter, i.e. in {{FileSystemBlobStore}}, there is no guarantee that a directory
will not be deleted concurrently (from a {{delete}} method) between its creation and writing
a file (in a {{get}} method):
> * the {{delete}} method for name-addressable blobs always deletes the job-specific storage
directory if there is no further blob for this job
> * the content-addressable blobs do that similarly but are shared among jobs and thus
only delete directories if there is no other blob.
> Since name-addressable blobs have not been used so far and the latter case typically
does not occur concurrently with get/put requests, this has not been a problem so far but
is more relevant after applying FLINK-6046.

This message was sent by Atlassian JIRA

View raw message