libcloud-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paul Tiplady (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LIBCLOUD-835) Malformed auth token causes fatal exception in Google Storage driver
Date Fri, 22 Jul 2016 18:38:20 GMT

    [ https://issues.apache.org/jira/browse/LIBCLOUD-835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15390017#comment-15390017
] 

Paul Tiplady commented on LIBCLOUD-835:
---------------------------------------

I neglected to update, that this issue didn't actually clear when I upgraded to 1.1.0.

I added some debug logging to the token generation code, and I have figured out what the problem
is.

Here's a sequence of token writes (W) and reads (R):

{code}
W: {"token_type": "Bearer", "expires_in": 3599, "expire_time": "2016-07-22T07:12:46Z", "access_token":
"ya29.<snip 107 chars>"}
W: {"token_type": "Bearer", "expires_in": 3294, "expire_time": "2016-07-22T18:17:05Z", "access_token":
"ya29.<snip 82 chars>"}
R: {"token_type": "Bearer", "expires_in": 3294, "expire_time": "2016-07-22T18:17:05Z", "access_token":
"ya29.<82 chars from new token"}<23 chars from original token>"}
{code}

The write_token code is not clearing the old token before writing the new one, resulting in
corruption.

The offending code, in libcloud/common/google.py:

{code}
        with os.fdopen(os.open(filename, os.O_CREAT | os.O_WRONLY,
                               int('600', 8)), 'w') as f:
            f.write(data)
{code}

In the case where the file exists, O_CREAT is a no-op, so we're just opening the existing
file and writing our bytes into it, without first clearing it. Need to set O_TRUNC as well,
to get > semantics instead of >>.

I don't know what causes different token lengths to be returned by the Google APIs; since
this issue appeared spontaneously, and only on one of my projects, it may be that the Google
auth APIs changed recently, thus triggering this latent bug. Also note that I'm using auth
type GoogleAuthType.GCE, which might scope the issue more tightly -- but the affected code
is used by all Google auth types, so this in principle could be hit on any of them.

Until it's understood under what circumstances the Google APIs return different token lengths,
I think it's safest to assume that this issue can break everybody using this library with
Google Cloud Storage, so this looks like an exceptionally critical bug; I propose raising
to Blocker level and making an immediate bugfix release with a fix, and backporting to all
supported versions.

I'm going to knock together a quick fix on my fork of Libcloud, since it's not much code.
I'll push a fix out without UTs onto my staging environment, and look at tests later. Happy
to contribute this fix back. Will you want a test for this fix, even though it's a one-liner?

> Malformed auth token causes fatal exception in Google Storage driver
> --------------------------------------------------------------------
>
>                 Key: LIBCLOUD-835
>                 URL: https://issues.apache.org/jira/browse/LIBCLOUD-835
>             Project: Libcloud
>          Issue Type: Bug
>            Reporter: Paul Tiplady
>            Priority: Critical
>
> One of my Django instances has started hitting a libcloud error which is causing a fatal
exception, bringing down the instance.
> It looks like libcloud is writing invalid JSON into the auth token, which then causes
a JSON parse error when it is subsequently read back in.
> Here's the token that's written:
> {code}
> $ cat /root/.google_libcloud_auth.<project>
> {"access_token": "<redacted>", "token_type": "Bearer", "expire_time": "2016-07-12T16:45:09Z",
"expires_in": 3559}09Z", "expires_in": 3537}
> {code}
> Note the two "expires_in" keys, one with a nonsense value of `3559}09Z"`
> Environment:
> Python 3.4.4 
> apache-libcloud==1.0.0



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message