Return-Path: X-Original-To: apmail-libcloud-commits-archive@www.apache.org Delivered-To: apmail-libcloud-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 53AC4100DB for ; Fri, 15 Nov 2013 10:13:34 +0000 (UTC) Received: (qmail 51060 invoked by uid 500); 15 Nov 2013 10:13:32 -0000 Delivered-To: apmail-libcloud-commits-archive@libcloud.apache.org Received: (qmail 51028 invoked by uid 500); 15 Nov 2013 10:13:31 -0000 Mailing-List: contact commits-help@libcloud.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@libcloud.apache.org Delivered-To: mailing list commits@libcloud.apache.org Received: (qmail 50633 invoked by uid 99); 15 Nov 2013 10:13:23 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 Nov 2013 10:13:23 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 8BEAE82E672; Fri, 15 Nov 2013 10:13:22 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: tomaz@apache.org To: commits@libcloud.apache.org Date: Fri, 15 Nov 2013 10:13:22 -0000 Message-Id: <2e78f3df832643e7a357f0c2bc850197@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [1/5] git commit: Fix the following auth related things in the OpenStack classes: Updated Branches: refs/heads/trunk d449ffc5f -> 608839543 Fix the following auth related things in the OpenStack classes: - Correctly handle ex_force_auth_token argument - Correctly cache, re-use and expire auth tokens Also update affected tests and test fixtures. Reported by Michael Farrell, part of LIBCLOUD-428. Project: http://git-wip-us.apache.org/repos/asf/libcloud/repo Commit: http://git-wip-us.apache.org/repos/asf/libcloud/commit/a5648b34 Tree: http://git-wip-us.apache.org/repos/asf/libcloud/tree/a5648b34 Diff: http://git-wip-us.apache.org/repos/asf/libcloud/diff/a5648b34 Branch: refs/heads/trunk Commit: a5648b34e3c33549913201013133a48488598b76 Parents: 5d44ce1 Author: Tomaz Muraus Authored: Wed Nov 13 14:49:49 2013 +0100 Committer: Tomaz Muraus Committed: Wed Nov 13 16:00:33 2013 +0100 ---------------------------------------------------------------------- docs/compute/drivers/openstack.rst | 13 ++++ libcloud/common/openstack.py | 68 +++++++++++--------- .../compute/fixtures/openstack/_v1_1__auth.json | 2 +- .../compute/fixtures/openstack/_v2_0__auth.json | 2 +- .../openstack/_v2_0__auth_deployment.json | 2 +- .../fixtures/openstack/_v2_0__auth_lon.json | 2 +- libcloud/test/compute/test_openstack.py | 50 ++++++++------ .../test/dns/fixtures/rackspace/auth_1_1.json | 2 +- .../test/dns/fixtures/rackspace/auth_2_0.json | 2 +- .../fixtures/rackspace/_v2_0__auth.json | 2 +- .../fixtures/rackspace/auth_2_0.json | 2 +- .../fixtures/cloudfiles/_v2_0__auth.json | 2 +- 12 files changed, 90 insertions(+), 59 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/libcloud/blob/a5648b34/docs/compute/drivers/openstack.rst ---------------------------------------------------------------------- diff --git a/docs/compute/drivers/openstack.rst b/docs/compute/drivers/openstack.rst index bc61dc0..81ef8cd 100644 --- a/docs/compute/drivers/openstack.rst +++ b/docs/compute/drivers/openstack.rst @@ -78,6 +78,16 @@ component of the URL (``12345`` in the example bellow). 4. Skipping normal authentication flow and hitting the API endpoint directly using the ``ex_force_auth_token`` argument ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +This is an advanced use cases which assumes you manage authentication and token +retrieval yourself. + +If you use this argument, the driver won't hit authentication service and as +such, won't be aware of the token expiration time. + +This means auth token will be considered valid for the whole life time of the +driver instance and you will need to manually re-instantiate a driver with a new +token before the currently used one is about to expire. + .. literalinclude:: /examples/compute/openstack/force_auth_token.py :language: python @@ -109,6 +119,9 @@ between different requests. This means that driver will only hit authentication service and obtain auth token on the first request or if the auth token is about to expire. +As noted in the example 4 above, this doesn't hold true if you use +``ex_force_auth_token`` argument. + Troubleshooting --------------- http://git-wip-us.apache.org/repos/asf/libcloud/blob/a5648b34/libcloud/common/openstack.py ---------------------------------------------------------------------- diff --git a/libcloud/common/openstack.py b/libcloud/common/openstack.py index f75ac94..1f13eab 100644 --- a/libcloud/common/openstack.py +++ b/libcloud/common/openstack.py @@ -143,7 +143,7 @@ class OpenStackAuthConnection(ConnectionUserAndKey): :type force: ``bool`` """ if not force and self.auth_version in AUTH_VERSIONS_WITH_EXPIRES \ - and self._is_token_valid(): + and self.is_token_valid(): # If token is still valid, there is no need to re-authenticate return self @@ -284,11 +284,12 @@ class OpenStackAuthConnection(ConnectionUserAndKey): return self - def _is_token_valid(self): + def is_token_valid(self): """ - Return True if the current taken is already cached and hasn't expired - yet. + Return True if the current auth token is already cached and hasn't + expired yet. + :return: ``True`` if the token is still valid, ``False`` otherwise. :rtype: ``bool`` """ if not self.auth_token: @@ -303,7 +304,6 @@ class OpenStackAuthConnection(ConnectionUserAndKey): time_tuple_expires = expires.utctimetuple() time_tuple_now = datetime.datetime.utcnow().utctimetuple() - # TODO: Subtract some reasonable grace time period if time_tuple_now < time_tuple_expires: return True @@ -484,30 +484,52 @@ class OpenStackBaseConnection(ConnectionUserAndKey): ex_force_service_type=None, ex_force_service_name=None, ex_force_service_region=None): + super(OpenStackBaseConnection, self).__init__( + user_id, key, secure=secure, timeout=timeout) self._ex_force_base_url = ex_force_base_url self._ex_force_auth_url = ex_force_auth_url self._auth_version = self._auth_version or ex_force_auth_version + self._ex_force_auth_token = ex_force_auth_token self._ex_tenant_name = ex_tenant_name self._ex_force_service_type = ex_force_service_type self._ex_force_service_name = ex_force_service_name self._ex_force_service_region = ex_force_service_region - self._osa = None - - if ex_force_auth_token: - self.auth_token = ex_force_auth_token - if ex_force_auth_token and not ex_force_base_url: raise LibcloudError( 'Must also provide ex_force_base_url when specifying ' 'ex_force_auth_token.') + if ex_force_auth_token: + self.auth_token = ex_force_auth_token + if not self._auth_version: self._auth_version = AUTH_API_VERSION - super(OpenStackBaseConnection, self).__init__( - user_id, key, secure=secure, timeout=timeout) + auth_url = self._get_auth_url() + + if not auth_url: + raise LibcloudError('OpenStack instance must ' + + 'have auth_url set') + + osa = OpenStackAuthConnection(self, auth_url, self._auth_version, + self.user_id, self.key, + tenant_name=self._ex_tenant_name, + timeout=self.timeout) + self._osa = osa + + def _get_auth_url(self): + """ + Retrieve auth url for this instance using either "ex_force_auth_url" + constructor kwarg of "auth_url" class variable. + """ + auth_url = self.auth_url + + if self._ex_force_auth_url is not None: + auth_url = self._ex_force_auth_url + + return auth_url def get_service_catalog(self): if self.service_catalog is None: @@ -557,30 +579,18 @@ class OpenStackBaseConnection(ConnectionUserAndKey): OpenStack uses a separate host for API calls which is only provided after an initial authentication request. """ + osa = self._osa - if not self.auth_token: - aurl = self.auth_url - - if self._ex_force_auth_url is not None: - aurl = self._ex_force_auth_url - - if not aurl: - raise LibcloudError('OpenStack instance must ' + - 'have auth_url set') - - osa = OpenStackAuthConnection(self, aurl, self._auth_version, - self.user_id, self.key, - tenant_name=self._ex_tenant_name, - timeout=self.timeout) - - # may throw InvalidCreds, etc + # Token is not available or it has expired. Need to retrieve a new one. + if not self._ex_force_auth_token and not osa.is_token_valid(): + # This call may throw InvalidCreds, etc osa.authenticate() self.auth_token = osa.auth_token self.auth_token_expires = osa.auth_token_expires self.auth_user_info = osa.auth_user_info - # pull out and parse the service catalog + # Pull out and parse the service catalog osc = OpenStackServiceCatalog(osa.urls, ex_force_auth_version= self._auth_version) self.service_catalog = osc http://git-wip-us.apache.org/repos/asf/libcloud/blob/a5648b34/libcloud/test/compute/fixtures/openstack/_v1_1__auth.json ---------------------------------------------------------------------- diff --git a/libcloud/test/compute/fixtures/openstack/_v1_1__auth.json b/libcloud/test/compute/fixtures/openstack/_v1_1__auth.json index ab45d58..365551d 100644 --- a/libcloud/test/compute/fixtures/openstack/_v1_1__auth.json +++ b/libcloud/test/compute/fixtures/openstack/_v1_1__auth.json @@ -2,7 +2,7 @@ "auth": { "token": { "id": "aaaaaaaaaaaa-bbb-cccccccccccccc", - "expires": "2011-11-23T21:00:14-06:00" + "expires": "2031-11-23T21:00:14-06:00" }, "serviceCatalog": { "cloudFilesCDN": [ http://git-wip-us.apache.org/repos/asf/libcloud/blob/a5648b34/libcloud/test/compute/fixtures/openstack/_v2_0__auth.json ---------------------------------------------------------------------- diff --git a/libcloud/test/compute/fixtures/openstack/_v2_0__auth.json b/libcloud/test/compute/fixtures/openstack/_v2_0__auth.json index fa75970..086ed69 100644 --- a/libcloud/test/compute/fixtures/openstack/_v2_0__auth.json +++ b/libcloud/test/compute/fixtures/openstack/_v2_0__auth.json @@ -2,7 +2,7 @@ "access": { "token": { "id": "aaaaaaaaaaaa-bbb-cccccccccccccc", - "expires": "2011-11-23T21:00:14.000-06:00" + "expires": "2031-11-23T21:00:14.000-06:00" }, "serviceCatalog": [ { http://git-wip-us.apache.org/repos/asf/libcloud/blob/a5648b34/libcloud/test/compute/fixtures/openstack/_v2_0__auth_deployment.json ---------------------------------------------------------------------- diff --git a/libcloud/test/compute/fixtures/openstack/_v2_0__auth_deployment.json b/libcloud/test/compute/fixtures/openstack/_v2_0__auth_deployment.json index 9c59431..ae3ba4e 100644 --- a/libcloud/test/compute/fixtures/openstack/_v2_0__auth_deployment.json +++ b/libcloud/test/compute/fixtures/openstack/_v2_0__auth_deployment.json @@ -2,7 +2,7 @@ "access": { "token": { "id": "aaaaaaaaaaaa-bbb-cccccccccccccc", - "expires": "2011-11-23T21:00:14.000-06:00" + "expires": "2031-11-23T21:00:14.000-06:00" }, "serviceCatalog": [ { http://git-wip-us.apache.org/repos/asf/libcloud/blob/a5648b34/libcloud/test/compute/fixtures/openstack/_v2_0__auth_lon.json ---------------------------------------------------------------------- diff --git a/libcloud/test/compute/fixtures/openstack/_v2_0__auth_lon.json b/libcloud/test/compute/fixtures/openstack/_v2_0__auth_lon.json index 9ec07b1..e76b22c 100644 --- a/libcloud/test/compute/fixtures/openstack/_v2_0__auth_lon.json +++ b/libcloud/test/compute/fixtures/openstack/_v2_0__auth_lon.json @@ -2,7 +2,7 @@ "access": { "token": { "id": "aaaaaaaaaaaa-bbb-cccccccccccccc", - "expires": "2011-11-23T21:00:14.000-06:00" + "expires": "2031-11-23T21:00:14.000-06:00" }, "serviceCatalog": [ { http://git-wip-us.apache.org/repos/asf/libcloud/blob/a5648b34/libcloud/test/compute/test_openstack.py ---------------------------------------------------------------------- diff --git a/libcloud/test/compute/test_openstack.py b/libcloud/test/compute/test_openstack.py index d8c7840..e7edd3f 100644 --- a/libcloud/test/compute/test_openstack.py +++ b/libcloud/test/compute/test_openstack.py @@ -117,6 +117,7 @@ class OpenStackAuthConnectionTests(unittest.TestCase): # TODO refactor and move into libcloud/test/common def setUp(self): + OpenStackBaseConnection.auth_url = None OpenStackBaseConnection.conn_classes = (OpenStackMockHttp, OpenStackMockHttp) @@ -244,7 +245,7 @@ class OpenStackAuthConnectionTests(unittest.TestCase): for i in range(0, count): osa.authenticate(force=False) - self.assertEqual(mocked_auth_method.call_count, count) + self.assertEqual(mocked_auth_method.call_count, 1) # No force reauth, valid / non-expired token osa.auth_token = None @@ -270,22 +271,22 @@ class OpenStackAuthConnectionTests(unittest.TestCase): self.assertEqual(mocked_auth_method.call_count, 0) for i in range(0, count): - osa.authenticate(force=False) - if i == 0: osa.auth_token_expires = soon - self.assertEqual(mocked_auth_method.call_count, 5) + osa.authenticate(force=False) + + self.assertEqual(mocked_auth_method.call_count, 1) def _get_mock_connection(self, mock_http_class, auth_url=None): OpenStackBaseConnection.conn_classes = (mock_http_class, mock_http_class) - connection = OpenStackBaseConnection(*OPENSTACK_PARAMS) if auth_url is None: - connection.auth_url = "https://auth.api.example.com" - else: - connection.auth_url = auth_url + auth_url = "https://auth.api.example.com" + + OpenStackBaseConnection.auth_url = auth_url + connection = OpenStackBaseConnection(*OPENSTACK_PARAMS) connection._ex_force_base_url = "https://www.foo.com" connection.driver = OpenStack_1_0_NodeDriver(*OPENSTACK_PARAMS) @@ -300,6 +301,7 @@ class OpenStack_1_0_Tests(unittest.TestCase, TestCaseMixin): driver_klass = OpenStack_1_0_NodeDriver driver_args = OPENSTACK_PARAMS driver_kwargs = {} + #driver_kwargs = {'ex_force_auth_version': '1.0'} @classmethod def create_driver(self): @@ -315,10 +317,12 @@ class OpenStack_1_0_Tests(unittest.TestCase, TestCaseMixin): return "https://servers.api.rackspacecloud.com/v1.0/slug" self.driver_klass.connectionCls.get_endpoint = get_endpoint - self.driver_klass.connectionCls.conn_classes = ( - OpenStackMockHttp, OpenStackMockHttp) + self.driver_klass.connectionCls.conn_classes = (OpenStackMockHttp, + OpenStackMockHttp) self.driver_klass.connectionCls.auth_url = "https://auth.api.example.com" + OpenStackMockHttp.type = None + self.driver = self.create_driver() # normally authentication happens lazily, but we force it here self.driver.connection._populate_hosts_and_request_paths() @@ -333,7 +337,7 @@ class OpenStack_1_0_Tests(unittest.TestCase, TestCaseMixin): self.driver.connection._populate_hosts_and_request_paths() expires = self.driver.connection.auth_token_expires - self.assertEqual(expires.isoformat(), "2011-11-23T21:00:14-06:00") + self.assertEqual(expires.isoformat(), "2031-11-23T21:00:14-06:00") def test_auth(self): if self.driver.connection._auth_version == '2.0': @@ -654,7 +658,7 @@ class OpenStackMockHttp(MockHttpTestCase): def _v1_0_UNAUTHORIZED_MISSING_KEY(self, method, url, body, headers): headers = { 'x-server-management-url': 'https://servers.api.rackspacecloud.com/v1.0/slug', - 'x-auth-token': 'FE011C19-CF86-4F87-BE5D-9229145D7A06', + 'x-auth-tokenx': 'FE011C19-CF86-4F87-BE5D-9229145D7A06', 'x-cdn-management-url': 'https://cdn.clouddrive.com/v1/MossoCloudFS_FE011C19-CF86-4F87-BE5D-9229145D7A06'} return (httplib.NO_CONTENT, "", headers, httplib.responses[httplib.NO_CONTENT]) @@ -817,27 +821,30 @@ class OpenStack_1_1_Tests(unittest.TestCase, TestCaseMixin): clear_pricing_data() self.node = self.driver.list_nodes()[1] - def test_auth_token_is_set(self): - # change base url and trash the current auth token so we can - # re-authenticate + def _force_reauthentication(self): + """ + Trash current auth token so driver will be forced to re-authentication + on next request. + """ self.driver.connection._ex_force_base_url = 'http://ex_force_base_url.com:666/forced_url' self.driver.connection.auth_token = None self.driver.connection.auth_token_expires = None + self.driver.connection._osa.auth_token = None + self.driver.connection._osa.auth_token_expires = None + + def test_auth_token_is_set(self): + self._force_reauthentication() self.driver.connection._populate_hosts_and_request_paths() self.assertEqual( self.driver.connection.auth_token, "aaaaaaaaaaaa-bbb-cccccccccccccc") def test_auth_token_expires_is_set(self): - # change base url and trash the current auth token so we can - # re-authenticate - self.driver.connection._ex_force_base_url = 'http://ex_force_base_url.com:666/forced_url' - self.driver.connection.auth_token = None - self.driver.connection.auth_token_expires = None + self._force_reauthentication() self.driver.connection._populate_hosts_and_request_paths() expires = self.driver.connection.auth_token_expires - self.assertEqual(expires.isoformat(), "2011-11-23T21:00:14-06:00") + self.assertEqual(expires.isoformat(), "2031-11-23T21:00:14-06:00") def test_ex_force_base_url(self): # change base url and trash the current auth token so we can @@ -896,6 +903,7 @@ class OpenStack_1_1_Tests(unittest.TestCase, TestCaseMixin): 'ex_force_auth_token': 'preset-auth-token', 'ex_force_base_url': base_url } + driver = self.driver_type(*self.driver_args, **kwargs) driver.list_nodes() http://git-wip-us.apache.org/repos/asf/libcloud/blob/a5648b34/libcloud/test/dns/fixtures/rackspace/auth_1_1.json ---------------------------------------------------------------------- diff --git a/libcloud/test/dns/fixtures/rackspace/auth_1_1.json b/libcloud/test/dns/fixtures/rackspace/auth_1_1.json index 1b5f6c1..fb02cf7 100644 --- a/libcloud/test/dns/fixtures/rackspace/auth_1_1.json +++ b/libcloud/test/dns/fixtures/rackspace/auth_1_1.json @@ -2,7 +2,7 @@ "auth":{ "token":{ "id":"fooo-bar-fooo-bar-fooo-bar", - "expires":"2011-10-29T17:39:28.000-05:00" + "expires":"2031-10-29T17:39:28.000-05:00" }, "serviceCatalog":{ "cloudFilesCDN":[ http://git-wip-us.apache.org/repos/asf/libcloud/blob/a5648b34/libcloud/test/dns/fixtures/rackspace/auth_2_0.json ---------------------------------------------------------------------- diff --git a/libcloud/test/dns/fixtures/rackspace/auth_2_0.json b/libcloud/test/dns/fixtures/rackspace/auth_2_0.json index c2943f2..df7ba6f 100644 --- a/libcloud/test/dns/fixtures/rackspace/auth_2_0.json +++ b/libcloud/test/dns/fixtures/rackspace/auth_2_0.json @@ -2,7 +2,7 @@ "access": { "token": { "id": "aaaaaaaaaaaa-bbb-cccccccccccccc", - "expires": "2011-11-23T21:00:14.000-06:00" + "expires": "2031-11-23T21:00:14.000-06:00" }, "serviceCatalog": [ { http://git-wip-us.apache.org/repos/asf/libcloud/blob/a5648b34/libcloud/test/loadbalancer/fixtures/rackspace/_v2_0__auth.json ---------------------------------------------------------------------- diff --git a/libcloud/test/loadbalancer/fixtures/rackspace/_v2_0__auth.json b/libcloud/test/loadbalancer/fixtures/rackspace/_v2_0__auth.json index 9fc3835..a837fff 100644 --- a/libcloud/test/loadbalancer/fixtures/rackspace/_v2_0__auth.json +++ b/libcloud/test/loadbalancer/fixtures/rackspace/_v2_0__auth.json @@ -2,7 +2,7 @@ "access": { "token": { "id": "aaaaaaaaaaaa-bbb-cccccccccccccc", - "expires": "2011-11-23T21:00:14.000-06:00" + "expires": "2031-11-23T21:00:14.000-06:00" }, "serviceCatalog": [ { http://git-wip-us.apache.org/repos/asf/libcloud/blob/a5648b34/libcloud/test/loadbalancer/fixtures/rackspace/auth_2_0.json ---------------------------------------------------------------------- diff --git a/libcloud/test/loadbalancer/fixtures/rackspace/auth_2_0.json b/libcloud/test/loadbalancer/fixtures/rackspace/auth_2_0.json index 05edc47..569445f 100644 --- a/libcloud/test/loadbalancer/fixtures/rackspace/auth_2_0.json +++ b/libcloud/test/loadbalancer/fixtures/rackspace/auth_2_0.json @@ -2,7 +2,7 @@ "access": { "token": { "id": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", - "expires": "2012-03-14T08:10:14.000-05:00" + "expires": "2031-03-14T08:10:14.000-05:00" }, "serviceCatalog": [ { http://git-wip-us.apache.org/repos/asf/libcloud/blob/a5648b34/libcloud/test/storage/fixtures/cloudfiles/_v2_0__auth.json ---------------------------------------------------------------------- diff --git a/libcloud/test/storage/fixtures/cloudfiles/_v2_0__auth.json b/libcloud/test/storage/fixtures/cloudfiles/_v2_0__auth.json index b9cbb9d..3457fdc 100644 --- a/libcloud/test/storage/fixtures/cloudfiles/_v2_0__auth.json +++ b/libcloud/test/storage/fixtures/cloudfiles/_v2_0__auth.json @@ -2,7 +2,7 @@ "access": { "token": { "id": "aaaaaaaaaaaa-bbb-cccccccccccccc", - "expires": "2011-11-23T21:00:14.000-06:00" + "expires": "2031-11-23T21:00:14.000-06:00" }, "serviceCatalog": [ {