flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nico Kruber (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-6380) BlobService concurrency issues between delete and put/get methods
Date Tue, 25 Apr 2017 09:06:04 GMT
Nico Kruber created FLINK-6380:

             Summary: BlobService concurrency issues between delete and put/get methods
                 Key: FLINK-6380
                 URL: https://issues.apache.org/jira/browse/FLINK-6380
             Project: Flink
          Issue Type: Bug
          Components: Network
    Affects Versions: 1.3.0
            Reporter: Nico Kruber

{{BlobCache#deleteAll(JobID)}} deletes the job directory which is only created at the start
of {{BlobCache#getURL(BlobKey)}} which then relies on the directory being present.

This is not restricted to the {{BlobCache}}, though, but also affects the {{BlobServer}} in
two ways:
1) its own local storage and
2) its backing {{BlobStore}}

For the latter, i.e. in {{FileSystemBlobStore}}, there is no guarantee that a directory will
not be deleted concurrently (from a {{delete}} method) between its creation and writing a
file (in a {{get}} method):

* the {{delete}} method for name-addressable blobs always deletes the job-specific storage
directory if there is no further blob for this job
* the content-addressable blobs do that similarly but are shared among jobs and thus only
delete directories if there is no other blob.

Since name-addressable blobs have not been used so far and the latter case typically does
not occur concurrently with get/put requests, this has not been a problem so far but is more
relevant after applying FLINK-6046.

This message was sent by Atlassian JIRA

View raw message