mesos-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrei Sekretenko (Jira)" <j...@apache.org>
Subject [jira] [Commented] (MESOS-10092) Cannot pull image from docker registry which does not reply with 'scope'/'service' in WWW-Authenticate header
Date Mon, 03 Feb 2020 18:07:00 GMT

    [ https://issues.apache.org/jira/browse/MESOS-10092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17029146#comment-17029146
] 

Andrei Sekretenko commented on MESOS-10092:
-------------------------------------------

Briefly discussed with [~vinodkone] and [~abudnik]; we came to conclusion that the safest
approach to fix this is the following:

First, we query the repository the old way, with the manifest/blob URI.
If it replies with both three of service, scope and realm, we assume that they know what they
are doing and proceed with building auth server URI as before.

If the initial `WWW-Authenticate` is missing any of the three, then, instead of failing, we
re-send the initial query to the repository root URI (nvcr.io/v2 in this example), get the
realm from the response and compose the scope ourselves (like Docker does).

> Cannot pull image from docker registry which does not reply with 'scope'/'service' in
WWW-Authenticate header
> -------------------------------------------------------------------------------------------------------------
>
>                 Key: MESOS-10092
>                 URL: https://issues.apache.org/jira/browse/MESOS-10092
>             Project: Mesos
>          Issue Type: Bug
>            Reporter: Andrei Sekretenko
>            Assignee: Andrei Sekretenko
>            Priority: Critical
>
> This problem was encountered when trying to specify container image nvcr.io/nvidia/tensorflow:19.12-tf1-py3
> When initiating Docker Registry authentication (https://docs.docker.com/registry/spec/auth/token/)
with nvcr.io, Mesos URI fetcher receives 'WWW-Authenticate' header without 'service' and 'scope'
params, and fails here:
> https://github.com/apache/mesos/blob/1e9b121273a6d9248a78ab44798bd4c1138c31ee/src/uri/fetchers/docker.cpp#L1083
> This is an example of an unsuccessful request made by Mesos:
> {code}
> curl -s -S -L -i --raw --http1.1 -H "Accept: application/vnd.docker.distribution.manifest.v2+json,application/vnd.docker.distribution.manifest.v1+json,application/vnd.docker.distribution.manifest.v1+prettyjws"
-y 60 https://nvcr.io/v2/nvidia/tensorflow/manifests/19.08-py3
> HTTP/1.1 401 Unauthorized
> Content-Type: text/html
> Date: Wed, 22 Jan 2020 19:01:57 GMT
> Server: nginx/1.14.2
> Www-Authenticate: Bearer realm="https://nvcr.io/proxy_auth?scope=repository:nvidia/tensorflow:pull,push"
> Content-Length: 195
> Connection: keep-alive
> <html>
> <head><title>401 Authorization Required</title></head>
> <body bgcolor="white">
> <center><h1>401 Authorization Required</h1></center>
> <hr><center>nginx/1.14.2</center>
> </body>
> </html>
> {code}
> At the same time, docker is perfectly capable of pulling this image.
> Note that the document "Token Authentication Specification" (https://docs.docker.com/registry/spec/auth/token/),
on which the Mesos implementation is based, is vague on the issue of registries that do not
provide  'scope'/'service' in WWW-Authenticate header.
> What Docker does differently (at the very least, in the case of nvcr.io):
> It sends the initial request not to the maniferst/blob URI, but to the repository root
URI (http:://nvcr.io/v2 in this case):
> {code}
> GET /v2/ HTTP/1.1
> Host: nvcr.io
> User-Agent: docker/18.03.1-ce go/go1.9.5 git-commit/9ee9f402cd kernel/4.15.0-60-generic
os/linux arch/amd64 UpstreamClient(Docker-Client/18.09.7 \(linux\))
> {code}
> To this, it receives response with a "realm" that contains no query arguments:
> {code}
> HTTP/1.1 401 Unauthorized
> Connection: close
> Content-Length: 195
> Content-Type: text/html
> Date: Wed, 29 Jan 2020 12:22:43 GMT
> Server: nginx/1.14.2
> Www-Authenticate: Bearer realm="https://nvcr.io/proxy_auth
> {code}
> Then, it composes the scope using the image ref and a hardcoded "pull" action: 
> https://github.com/docker/distribution/blob/a8371794149d1d95f1e846744b05c87f2f825e5a/registry/client/auth/session.go#L174
> (in a full accordance with this spec: https://docs.docker.com/registry/spec/auth/scope/)
> and sends the following request to  https://nvcr.io/proxy_auth :
> {code}
> GET /proxy_auth?scope=repository%3Anvidia%2Ftensorflow%3Apull HTTP/1.1
> Host: nvcr.io
> User-Agent: Go-http-client/1.1
> {code}
> (Note that 'push' is absent from the scope)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message