jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexander Klimetschek (Jira)" <j...@apache.org>
Subject [jira] [Comment Edited] (OAK-8552) Minimize network calls required when creating a direct download URI
Date Tue, 27 Aug 2019 06:20:00 GMT

    [ https://issues.apache.org/jira/browse/OAK-8552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16916411#comment-16916411
] 

Alexander Klimetschek edited comment on OAK-8552 at 8/27/19 6:19 AM:
---------------------------------------------------------------------

To add some context:

In our case that uncovered this issue, the problem really only exists because in some special
case, we uploaded binaries through JCR with async uploads in caching DS enabled, and then
immediately after the session.save() requested presigned GET URLs to pass them on to an external
service. That lead us first to make the presigned URL generation support existence check (OAK-7998,
plus the implicit check in getReference()) and „polling“ by returning null for not-yet-in-blob-store
cases. However, that is shifting the solution to the wrong end and increasing application
complexity (polling loop).

(Also note this edge case is a short term solution to be replaced at some point with proper
direct binary access for upload)

The source of the problem here is the async upload: we need to switch this to synchronous
uploads = blocking session.save(), to avoid the issue in the first place.

In all regular cases, binaries are uploaded through the new direct binary access, which by
design guarantees the presence of the binary when the reference is added to the NodeStore.

If the binary gets deleted from the blob store due to some malfunctioning, then it does not
matter to the application if we return null or an URL that returns 404. The binary is not
available either way. But the latter allows us to completely drop any existence checks upon
presigned GET URL generation.

Same for inlined: configuration must prevent this in the first place, then no special check
is required at access time.


was (Author: alexander.klimetschek):
To add some context:

In our case that uncovered this issue, the problem really only exists because in some special
case, we uploaded binaries through JCR with async uploads in caching DS enabled, and then
immediately after the session.save() requested presigned GET URLs to pass them on to an external
service. That lead us first to make the presigned URL generation support existence check and
„polling“ by returning null for not-yet-in-blob-store cases. However, that is shifting
the solution to the wrong end and increasing application complexity (polling loop).

(Also note this edge case is a short term solution to be replaced at some point with proper
direct binary access for upload)

The source of the problem here is the async upload: we need to switch this to synchronous
uploads = blocking session.save(), to avoid the issue in the first place.

In all regular cases, binaries are uploaded through the new direct binary access, which by
design guarantees the presence of the binary when the reference is added to the NodeStore.

If the binary gets deleted from the blob store due to some malfunctioning, then it does not
matter to the application if we return null or an URL that returns 404. The binary is not
available either way. But the latter allows us to completely drop any existence checks upon
presigned GET URL generation.

Same for inlined: configuration must prevent this in the first place, then no special check
is required at access time.

> Minimize network calls required when creating a direct download URI
> -------------------------------------------------------------------
>
>                 Key: OAK-8552
>                 URL: https://issues.apache.org/jira/browse/OAK-8552
>             Project: Jackrabbit Oak
>          Issue Type: Sub-task
>          Components: blob-cloud, blob-cloud-azure
>            Reporter: Matt Ryan
>            Assignee: Matt Ryan
>            Priority: Major
>         Attachments: OAK-8552_ApiChange.patch
>
>
> We need to isolate and try to optimize network calls required to create a direct download
URI.



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Mime
View raw message