Mailing-List: contact user-help@jclouds.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@jclouds.apache.org
MIME-Version: 1.0
In-Reply-To: <678e8d0c-907f-23c0-4d4a-20e5ca910dd0@enterprisedb.com>
References: <d88d61c5-22c1-a797-7e40-0aade7ab2d4d@enterprisedb.com>
 <CANyjYNZvHs2q=k5MZY6cSLE6i0wEWVQixCzndTwQz4kUGuv2mg@mail.gmail.com> <678e8d0c-907f-23c0-4d4a-20e5ca910dd0@enterprisedb.com>
From: Ignasi Barrera <nacx@apache.org>
Date: Mon, 9 Jan 2017 17:07:59 +0100
Message-ID: <CANyjYNbBEYeRS5b1w6e1Kr4KV44JswfM2-LU8+xXpGk=6kNheQ@mail.gmail.com>
Subject: Re: OpenStack identity endpoint and serviceCatalog usage
To: "user@jclouds.apache.org" <user@jclouds.apache.org>
Content-Type: text/plain; charset=UTF-8
archived-at: Mon, 09 Jan 2017 16:08:23 -0000

Uops, sorry :)

https://blueprints.launchpad.net/keystone/+spec/service-catalog-version-support

I don't know if this was actually implemented in any of the old
OpenStack versions and if they are returning it now. I hope other
OpenStack folks in this link can clarify this. I saw the usage of the
"versionId" field in the code, and tried to find were it came from.

On 9 January 2017 at 17:03, Ryan Shoemaker
<ryan.shoemaker@enterprisedb.com> wrote:
> Thanks Ignassi for confirming my suspicions.
>
> I think you forgot to include the blueprint [1] reference you mentioned
> below...
>
> --Ryan
>
>
> On Jan9 9:00 AM, Ignasi Barrera wrote:
>>
>> Hi Ryan,
>>
>> Apologies for the late reply. The problem here is that no v2.0
>> identity endpoint is returned in the service catalog. As you say,
>> jclouds does the version discovery and it can only work with the
>> endpoints returned in the service catalog. An initial request is
>> performed to authenticate the user to the endpoint configured in the
>> ContextBuilder, and all subsequent operations are done against the
>> corresponding endpoint discovered from the service catalog. That's why
>> the second request to keystone is done to the v3 endpoint.
>>
>> I'm not an OpenStack expert, and i don't know how OpenStack should be
>> configured to make the v2 endpoint part of the service catalog
>> response, but that needs to happen in order to let jclouds discover
>> the version it needs.
>>
>> Apart from that (and for completeness), if several endpoints are
>> returned for the same service, jclouds will pick the first one,
>> unless:
>>
>> * A "versionId" field comes in the endpoint object in the json
>> response (according to this blueprint [1]; I don't know the state of
>> the art of Openstack and if that field is still returned or not).
>> * AND you use the "ContextBuilder.apiVersion" method to force the
>> version you want.
>>
>>
>> HTH,
>>
>> I.
>>
>> On 5 January 2017 at 20:33, Ryan Shoemaker
>> <ryan.shoemaker@enterprisedb.com> wrote:
>>>
>>> Hi,
>>>
>>> I'm hoping to get a better understanding of how jclouds discovers and
>>> decides which identity endpoint to use.  Feel free to redirect me to docs
>>> if
>>> this is already written up somewhere.
>>>
>>> In my case, we have a community Mitaka installation running and an app
>>> that
>>> uses jclouds v2.0.0.  Initially, nothing identity related was working
>>> because Mitaka wasn't configured to respond to V2.0 Identity API requests
>>> -
>>> only V3, which isn't supported by jclouds yet.  I didn't setup/configure
>>> this installation of OpenStack, so I'm not sure if that's a common issue
>>> or
>>> not - I don't know if the V2.0 Identity API is normally turned off in a
>>> fresh install or what.  It's entirely possible that there's something
>>> misconfigured in there causing the behavior I describe below.  Anyway,
>>> once
>>> the V2.0 Identity endpoint was enabled, I turned on tracing and can see
>>> that
>>> even though I create my context builder explicitly using the V2.0
>>> endpoint
>>> URL, jclouds switches over and uses whatever comes back in the
>>> 'serviceCatalog' param in the token response.  In my case, that happens
>>> to
>>> point to the V3 Identity endpoint.  At that point, jclouds fails to work
>>> because it is making V2.0 requests on the V3 endpoints.  Here's some
>>> tracing
>>> showing that in more detail:
>>>
>>>
>>> FINE: >>
>>>
>>> "{"auth":{"passwordCredentials":{"username":"someuser","password":"somepass"},"tenantName":"sometenant"}}"
>>> FINE: >> POST http://mitaka-host.com:5000/v2.0/tokens HTTP/1.1
>>> FINE: >> Accept: application/json
>>> FINE: >> Content-Type: application/json
>>> FINE: >> Content-Length: 108
>>>
>>> FINE: << HTTP/1.1 200 OK
>>> FINE: << Vary: X-Auth-Token
>>> FINE: << Date: Wed, 04 Jan 2017 20:45:54 GMT
>>> FINE: << Keep-Alive: timeout=5, max=97
>>> FINE: << Connection: Keep-Alive
>>> FINE: << x-openstack-request-id: req-390cebba-7cec-472c-88f4-8be6f6a0c236
>>> FINE: << Server: Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5
>>> FINE: << Content-Type: application/json
>>> FINE: << Content-Length: 3620
>>> FINE: <<
>>> "{"access":{
>>>     "token":{
>>>        "issued_at":"2017-01-04T20:45:54.000000Z",
>>>        "expires":"2017-01-04T21:45:54Z",
>>>
>>>
>>> "id":"gAAAAABYbV8C5pTf30IRs7ybxmbflLk539RDldDU6Q7DBHTTXTY79pabgotPSfy41Nt2nZ82isP0RxsfRuxWRlb1fnlWy1E8zlQdM3xZbrkjP27gSlmLcsV298v-CH8R4GY7wnIvfJjO0MO4DMAgFvvMIA12ByG4kqIJHZ92VV_F6Vby33gR574",
>>>        "tenant":{
>>>           "description":"somedescription",
>>>           "enabled":true,
>>>           "id":"6408ffc2d9034498bc764c230a90e591",
>>>           "name":"sometenant"
>>>        },
>>>        "audit_ids":[
>>>           "sEaG03ZdRXSeRMOr9GVCDA"
>>>        ]
>>>     },
>>>     "serviceCatalog":[
>>>        {
>>>           "endpoints":[
>>>              {
>>>
>>>
>>> "adminURL":"http://mitaka-host.com:8774/v2.1/6408ffc2d9034498bc764c230a90e591",
>>>                 "region":"uk",
>>>
>>>
>>> "internalURL":"http://mitaka-host.com:8774/v2.1/6408ffc2d9034498bc764c230a90e591",
>>>                 "id":"9d531ab7259e44738c953ec8766b0bc4",
>>>
>>>
>>> "publicURL":"http://mitaka-host.com:8774/v2.1/6408ffc2d9034498bc764c230a90e591"
>>>              }
>>>           ],
>>>           "endpoints_links":[
>>>
>>>           ],
>>>           "type":"compute",
>>>           "name":"nova"
>>>        },
>>>        {
>>>           "endpoints":[
>>>              {
>>>                 "adminURL":"http://mitaka-host.com:9696",
>>>                 "region":"uk",
>>>                 "internalURL":"http://mitaka-host.com:9696",
>>>                 "id":"1312e551abfe4c6d886e1a471f92bddd",
>>>                 "publicURL":"http://mitaka-host.com:9696"
>>>              }
>>>           ],
>>>           "endpoints_links":[
>>>
>>>           ],
>>>           "type":"network",
>>>           "name":"neutron"
>>>        },
>>>        {
>>>           "endpoints":[
>>>              {
>>>
>>>
>>> "adminURL":"http://mitaka-host.com:8776/v2/6408ffc2d9034498bc764c230a90e591",
>>>                 "region":"uk",
>>>
>>>
>>> "internalURL":"http://mitaka-host.com:8776/v2/6408ffc2d9034498bc764c230a90e591",
>>>                 "id":"02fb1ff3b41a40ceb4cb28abfa5bd547",
>>>
>>>
>>> "publicURL":"http://mitaka-host.com:8776/v2/6408ffc2d9034498bc764c230a90e591"
>>>              }
>>>           ],
>>>           "endpoints_links":[
>>>
>>>           ],
>>>           "type":"volumev2",
>>>           "name":"cinderv2"
>>>        },
>>>        {
>>>           "endpoints":[
>>>              {
>>>                 "adminURL":"http://mitaka-host.com:9292",
>>>                 "region":"uk",
>>>                 "internalURL":"http://mitaka-host.com:9292",
>>>                 "id":"3681888e8b62421f81e93d4270201520",
>>>                 "publicURL":"http://mitaka-host.com:9292"
>>>              }
>>>           ],
>>>           "endpoints_links":[
>>>
>>>           ],
>>>           "type":"image",
>>>           "name":"glance"
>>>        },
>>>        {
>>>           "endpoints":[
>>>              {
>>>
>>>
>>> "adminURL":"http://mitaka-host.com:8776/v1/6408ffc2d9034498bc764c230a90e591",
>>>                 "region":"uk",
>>>
>>>
>>> "internalURL":"http://mitaka-host.com:8776/v1/6408ffc2d9034498bc764c230a90e591",
>>>                 "id":"252797f5a4bb44c0844bff5586a23e16",
>>>
>>>
>>> "publicURL":"http://mitaka-host.com:8776/v1/6408ffc2d9034498bc764c230a90e591"
>>>              }
>>>           ],
>>>           "endpoints_links":[
>>>
>>>           ],
>>>           "type":"volume",
>>>           "name":"cinder"
>>>        },
>>>        {
>>>           "endpoints":[
>>>              {
>>>                 "adminURL":"http://mitaka-host.com:8080/v1",
>>>                 "region":"uk",
>>>
>>>
>>> "internalURL":"http://mitaka-host.com:8080/v1/AUTH_6408ffc2d9034498bc764c230a90e591",
>>>                 "id":"32f4fa49d90e4ad58b2b694bc1aab1ed",
>>>
>>>
>>> "publicURL":"http://mitaka-host.com:8080/v1/AUTH_6408ffc2d9034498bc764c230a90e591"
>>>              }
>>>           ],
>>>           "endpoints_links":[
>>>
>>>           ],
>>>           "type":"object-store",
>>>           "name":"swift"
>>>        },
>>>        {
>>>           "endpoints":[
>>>              {
>>>                 "adminURL":"http://mitaka-host.com:35357/v3",
>>>                 "region":"uk",
>>>                 "internalURL":"http://mitaka-host.com:5000/v3",
>>>                 "id":"23ace8e63eab431d81ccf5bfabc1ad53",
>>>                 "publicURL":"http://mitaka-host.com:5000/v3"
>>>              }
>>>           ],
>>>           "endpoints_links":[
>>>
>>>           ],
>>>           "type":"identity",
>>>           "name":"keystone"
>>>        }
>>>     ],
>>>     "user":{
>>>        "username":"someuser",
>>>        "roles_links":[
>>>
>>>        ],
>>>        "id":"a0da46b26af74a44a7c89cab6d9a6439",
>>>        "roles":[
>>>           {
>>>              "name":"admin"
>>>           }
>>>        ],
>>>        "name":"someuser"
>>>     },
>>>     "metadata":{
>>>        "is_admin":0,
>>>        "roles":[
>>>           "dca7c9a77e294f97a619e2b7da246550"
>>>        ]
>>>     }
>>> }
>>> }"
>>>
>>>
>>> FINE: >> GET http://mitaka-host.com:35357/v3/tenants?name=sometenant
>>> HTTP/1.1
>>> FINE: >> Accept: application/json
>>> FINE: >> X-Auth-Token:
>>>
>>> gAAAAABYbV8C5pTf30IRs7ybxmbflLk539RDldDU6Q7DBHTTXTY79pabgotPSfy41Nt2nZ82isP0RxsfRuxWRlb1fnlWy1E8zlQdM3xZbrkjP27gSlmLcsV298v-CH8R4GY7wnIvfJjO0MO4DMAgFvvMIA12ByG4kqIJHZ92VV_F6Vby33gR574
>>>
>>> FINE: << HTTP/1.1 404 Not Found
>>> FINE: << Vary: X-Auth-Token
>>> FINE: << Date: Wed, 04 Jan 2017 20:45:55 GMT
>>> FINE: << Keep-Alive: timeout=5, max=100
>>> FINE: << Connection: Keep-Alive
>>> FINE: << x-openstack-request-id: req-94bea180-8447-485a-8485-bf3f57e8d225
>>> FINE: << Server: Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5
>>> FINE: << Content-Type: application/json
>>> FINE: << Content-Length: 93
>>> FINE: << "{"error": {"message": "The resource could not be found.",
>>> "code":
>>> 404, "title": "Not Found"}}"
>>>
>>>
>>> Again, even though I created the jclouds ContextBuilder with the V2.0
>>> endpoint, it looks like jclouds read the identity admin endpoint from the
>>> 'serviceCatalog' in the response and then switched to using it, even
>>> though
>>> it is a V3 endpoint.  Is this normal expected behavior for jclouds?  I'm
>>> guessing it has more to do with how Mitaka is configured, but I don't see
>>> how jclouds would be able to talk to newer versions of OpenStack unless
>>> they
>>> disable the V3 Identity endpoint or configure it in such a way that the
>>> V2.0
>>> endpoint is exposed for jclouds to see it.  Are there any READMEs on how
>>> to
>>> use jclouds keystone APIs with newer versions of OpenStack?
>>>
>>> In general, it seems like OpenStack is moving more towards a model where
>>> API
>>> versioning is supposed to be discovered by clients.  For example:
>>> https://review.openstack.org/#/c/130669/, specifically the patch comment
>>> that says "if some one is using third party client, it will break. Only
>>> keystone client handles version discovery.    Keystone doesn't even
>>> document
>>> how to handle version discovery, so any third party clients such as
>>> jcloud
>>> will likely fail"
>>>
>>> Is there any update on jclouds adding support for the V3 Identity API?
>>>
>>> Thanks,
>>>
>>> --Ryan
>
>
>