beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jacob Marble (JIRA)" <>
Subject [jira] [Commented] (BEAM-3638) S3 occasionally fails to delete
Date Wed, 07 Feb 2018 17:21:00 GMT


Jacob Marble commented on BEAM-3638:

I believe this is a mistake. The "subsequent errors" are emitted by Filesystems.copy(). Closing
this for now, will re-open if I find something concrete.

> S3 occasionally fails to delete
> -------------------------------
>                 Key: BEAM-3638
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-extensions
>    Affects Versions: 2.3.0
>            Reporter: Jacob Marble
>            Assignee: Jacob Marble
>            Priority: Critical
>             Fix For: 2.3.0
> When AmazonS3Client.deleteObjects() is called, the S3 service occasionally returns a
retryable error, and also deletes the object. Beam retries the request, and the S3 service
correctly returns a "404 Not Found" error. The job fails needlessly.
> I propose to fix this by logging and allowing "404 Not Found" errors; if the object doesn't
exist then the current state is also the desired state. Should have a PR later today.
> The first error looks like this:
> {{ One
or more objects could not be deleted (Service: null; Status Code: 200; Error Code: null; Request
ID: [redacted]; S3 Extended Request ID: [redacted]), S3 Extended Request ID: [redacted]}}
> Subsequent errors:
> {{ Not Found (Service:
Amazon S3; Status Code: 404; Error Code: 404 Not Found; Request ID: [redacted]; S3 Extended
Request ID: [redacted]), S3 Extended Request ID: [redacted]}}

This message was sent by Atlassian JIRA

View raw message