Mailing-List: contact cloudstack-users-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-users@incubator.apache.org
Received-SPF: pass (athena.apache.org: domain of heince@gmail.com designates
 209.85.210.47 as permitted sender)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1084)
Subject: Re: (SOLVED] Having trouble getting a CloudStack API URL with api_key
 and generated signature to work from the browser (Firefox).
From: heince kurniawan <heince@gmail.com>
In-Reply-To: 
 <93099572B72EB341B81A644E134F240B012F6C818194@SJCPMAILBOX01.citrite.net>
Date: Tue, 19 Jun 2012 22:28:09 +0800
Content-Transfer-Encoding: quoted-printable
Message-Id: <3E52F371-9143-44F0-A39C-C2D9607E66BB@gmail.com>
References: 
 <93099572B72EB341B81A644E134F240B012F6C818194@SJCPMAILBOX01.citrite.net>
To: cloudstack-users@incubator.apache.org

Nice,

but since you split it by '&' , i expect a problem when specifying =
ldapConfig 'queryfilter' let say =3D> (&(uid=3D%u))

Regards,
Heince


On 19-Jun-2012, at 12:09 PM, Evan Miller wrote:

> Hi Anatha (and Alena and others):
>=20
> Thanks for your script.
> I ended up finding this gem of a perl script:
>   =
https://github.com/snumano/CloudStack-API/blob/master/generate-url.pl
> in this really excellent article about signed and (mostly) unsigned =
api calls:
>   =
http://www.shapeblue.com/2012/05/10/using-the-api-for-advanced-network-man=
agement/
> I used that gem of a script for my own perl script below:
>=20
> #!/usr/bin/perl -w
>=20
> use strict;
> use warnings;
>=20
> use Digest::SHA qw(hmac_sha1);
> use Encode;
> use Getopt::Std;
> use File::Basename qw(basename);
> use HTTP::Request;
> use JSON::XS;
> use LWP::UserAgent;
> use MIME::Base64;
> use URI::Encode;
>=20
> my ($content, $debug, $field, $value);
>=20
> my $my_filename =3D basename($0, '');
>=20
> our ($opt_a, $opt_d, $opt_f, $opt_s, $opt_u);
>=20
> $MY_CS_IP_AND_PORT =3D 'Enter your CS Server Name or IP here followed =
by :8080';
>=20
> my $site =3D "http://$MY_CS_IP_AND_PORT/client/api?";
>=20
> getopt "adfsu";
>=20
> if (!defined ($opt_u) || !defined ($opt_a) || !defined ($opt_s) || =
!defined ($opt_f)){
>    my $usage =3D "Usage:\n$my_filename -d <0|1> -f <flag: 1. URL ";
>    $usage =3D "$usage" . "only, OR 2. URL and Output> -u =
\"command=3D<cmd>\" ";
>    $usage =3D "$usage" .  "-a <api_key> -s <secret_key>\n";
>    die ("$usage");
> }
>=20
> ######################################################################
> # Process input parameters.
> ######################################################################
>=20
> my $api_key =3D $opt_a;
> $debug =3D $opt_d;
> my $secret_key =3D $opt_s;
> my $command =3D $opt_u;
>=20
> # Flag for output: 1. URL only, 2. Both URL and Output
> my $flag =3D $opt_f;
>=20
> # URL encoded for special characters.
> my $uri =3D URI::Encode->new();
>=20
> ######################################################################
> ### Step 1
> ### Combine parts to form initial URL.
> ######################################################################
>=20
> my $query =3D $command."&apiKey=3D".$api_key;
> my @list =3D split (/&/, $query);
> foreach (@list){
>    if (/(.+)\=3D(.+)/) {
>        $field =3D $1;
>        # Encode_reserved option is set to 1.
>        $value =3D $uri->encode($2, 1);
>        $_ =3D $field."=3D".$value;
>    }
> }
>=20
> ######################################################################
> ### Step 2
> ### Lower-case each parameter and parameter value.
> ######################################################################
>=20
> foreach (@list){
>    $_ =3D lc ($_);
> }
>=20
> ######################################################################
> ### Step 3
> # Sort parameter=3Dparameter_values by parameter and join each
> # with an ampersand.
> ######################################################################
>=20
> my $output =3D join ("&", sort @list);
>=20
> ######################################################################
> ### Step 4
> ### HMAC SHA1 encode the URL.
> ######################################################################
>=20
> my $digest =3D hmac_sha1($output, $secret_key);
> my $base64_encoded =3D encode_base64 ($digest);
> chomp($base64_encoded);
>=20
> # Encode_reserved option is set to 1.
> my $url_encoded =3D $uri->encode($base64_encoded, 1);
>=20
> ######################################################################
> ### Step 5
> ### Combine parts for the Final URL.
> ######################################################################
>=20
> my $final_url =3D =
$site."apikey=3D".$api_key."&".$command."&signature=3D".$url_encoded;
>=20
> ######################################################################
> ### Step 6
> ### Print out the Final URL and/or the output from
> ### executing the Final URL.
> ######################################################################
>=20
> if ($flag =3D=3D 1 || $flag =3D=3D 2){
>    if (defined $debug) {
>        if ($debug !=3D 0) {
>            print "FINAL URL:\n";
>        }
>    }
>    print "$final_url";
>    if ($flag =3D=3D 1){
>        exit;
>    }
>    if ($flag =3D=3D 2) {
>        my $agent =3D LWP::UserAgent->new(env_proxy =3D> 1,keep_alive =
=3D> 1, timeout =3D> 30);
>        my $header =3D HTTP::Request->new(GET =3D> $final_url);
>        my $request =3D HTTP::Request->new('GET', $final_url, $header);
>        my $response =3D $agent->request($request);
>=20
>        # Check the outcome of the response
>        if ($response->is_success){
>            if (defined $debug) {
>                if ($debug !=3D 0) {
>                    print "\nHEADERS:\n";
>                    print $response->headers_as_string;
>                }
>            }
>            print "\nCONTENT:\n";
>            $content =3D $response->as_string;
>            print $content;
>            exit 0;
>        } elsif ($response->is_error){
>            print "\nError My Final URL: $final_url\n";
>            print $response->error_as_HTML;
>            exit 1;
>        }
>    }
> }
>=20
> Regards,
> Evan
>=20
>=20
>=20
> -----Original Message-----
> From: Anantha Kasetty [mailto:Anantha.Kasetty@citrix.com]=20
> Sent: Saturday, June 09, 2012 6:52 AM
> To: cloudstack-users@incubator.apache.org
> Subject: Re: Having trouble getting a CloudStack API URL with api_key =
and generated signature to work from the browser (Firefox).
>=20
> Had to dust up my long forgotten Perl skills,  here is a sample =
program
>=20
> use URI::Escape;
> use Digest::SHA qw(hmac_sha1_base64);
>=20
> sub hmac_digest {
>    my $uri =3D shift @_;
>    my $key =3D shift @_;
>=20
>    $digest =3D hmac_sha1_base64($uri, $key);
>=20
>    #@#$%! perl does not pad the output of base64
>=20
>    while (length($digest) % 4) {
>        $digest .=3D '=3D';
>    }  =20
>    return $digest;
> }
>=20
> sub escape_hash {
>    my %hash =3D @_;=20
>    my @pairs;
>=20
>    for my $key (sort keys %hash) {
>        push @pairs, join "=3D", map { uri_escape($_) } lc($key), =
lc($hash{$key});
>    }  =20
>    return join "&", @pairs;
> }
>=20
> my $secret_key =3D
> =
'kNd2VxlXxCXwyJGlidr0ZcmcqXSH2refwxZTStD6If4vJu4QmJPOIui0rgr88mDI6DuGQTzP9=
e
> QNOjlZBTReKg' ;
>=20
>=20
> my $uri =3D escape_hash(
> =
'apikey'=3D>'8v_gevjjgdjbbhibmlle4yyhkseqhrefztnv4up2fu3k9y12th7lscsn6-7se=
ago
> 1ycctnute1ot0v7npcus8q',
>                       'command'=3D>'listZones',
>                       'response'=3D>'json');
>=20
> $signature =3D uri_escape(hmac_digest($uri, $secret_key));
>=20
> $uri .=3D "&signature=3D$signature";
> print "$uri\n";
>=20
>=20
>=20
> Let me know if this works for you.
>=20
>=20
>=20
> regards,
> Anantha Kasetty=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
>=20
> On 6/8/12 8:40 PM, "Alena Prokharchyk" <Alena.Prokharchyk@citrix.com>
> wrote:
>=20
>> On 6/8/12 7:08 PM, "Evan Miller" <Evan.Miller@citrix.com> wrote:
>>=20
>>> Hi Alena:
>>>=20
>>> -----Original Message-----
>>> From: Alena Prokharchyk [mailto:Alena.Prokharchyk@citrix.com]
>>> Sent: Friday, June 08, 2012 3:56 PM
>>> To: cloudstack-users@incubator.apache.org
>>> Subject: Re: Having trouble getting a CloudStack API URL with =
api_key and
>>> generated signature to work from the browser (Firefox).
>>>=20
>>> On 6/8/12 12:23 PM, "Evan Miller" <Evan.Miller@citrix.com> wrote:
>>>=20
>>>> Hi:
>>>>=20
>>>> I am not doing something quite right yet generating a good
>>>> CloudStack API URL.
>>>>=20
>>>> I still get this same error from the browser when
>>>> I try to execute the final url:
>>>>=20
>>>> { "listvirtualmachinesresponse" : {"errorcode" : 401, "errortext" :
>>>> "unable to verify user credentials"}  }
>>>>=20
>>>> Here is what my perl script is doing:
>>>>=20
>>>> Original Command String:
>>>>=20
>>>> =
apiKey=3D8v_GEvJJgDjbbHIBmlle4yyHKseQhRefztnv4UP2fU3K9y12TH7lscsn6-7SEaGO1=

>>>> y
>>>> C
>>>> CTnUtE1oT0v7npCuS8Q&command=3DlistVirtualMachines&response=3Djson
>>>>=20
>>>> Sorted (by field), lower-case Command String:
>>>>=20
>>>> =
apikey=3D8v_gevjjgdjbbhibmlle4yyhkseqhrefztnv4up2fu3k9y12th7lscsn6-7seago1=

>>>> y
>>>> c
>>>> ctnute1ot0v7npcus8q&command=3Dlistvirtualmachines&response=3Djson
>>>>=20
>>>> Encoded, sorted, lower-case Command String:
>>>>=20
>>>> =
apikey%3D8v_gevjjgdjbbhibmlle4yyhkseqhrefztnv4up2fu3k9y12th7lscsn6-7seag
>>>> o
>>>> 1
>>>> =
ycctnute1ot0v7npcus8q%26command%3Dlistvirtualmachines%26response%3Djson
>>>=20
>>> Here is the problem - you have to encode just parameter values, not =
the
>>> parameters themselves and definitely not "=3D" and "&" special =
chars.
>>>=20
>>> You can look at the java code sample I wrote for the dev bootcamp, =
here
>>> is
>>> the link in the source tree:
>>>=20
>>> =
http://git.cloud.com/cgit/cloudstack-oss/tree/test/src/com/cloud/test/dem
>>> o
>>> /Demo.java?h=3D3.0.x
>>>=20
>>>=20
>>> - Begin -
>>>=20
>>> I don't really know Java, but I did see your numbered steps
>>> at the bottom.
>>>=20
>>> I still get an error for the final url:
>>>=20
>>> { "listvirtualmachinesresponse" : {"errorcode" : 401, "errortext" :
>>> "unable to verify user credentials and/or request signature"}  }
>>>=20
>>> Here is what I tried ... continuing in perl.
>>> I encoded just the parameters (not the field values).
>>> The parameters didn't really require any encoding.
>>> No special characters, but I did it anyway.
>>> So, after encoding, the parameters didn't change.
>>> I did encode the signature before appending to the
>>> final url.
>>>=20
>>> Here's the flow ...
>>>=20
>>> Original Command String:
>>>=20
>>> =
apiKey=3D8v_GEvJJgDjbbHIBmlle4yyHKseQhRefztnv4UP2fU3K9y12TH7lscsn6-7SEaGO1=
y
>>> C
>>> CTnUtE1oT0v7npCuS8Q&command=3DlistVirtualMachines&response=3Djson
>>>=20
>>> Command String with Encoded Parameters:
>>>=20
>>> =
apiKey=3D8v_GEvJJgDjbbHIBmlle4yyHKseQhRefztnv4UP2fU3K9y12TH7lscsn6-7SEaGO1=
y
>>> C
>>> CTnUtE1oT0v7npCuS8Q&command=3DlistVirtualMachines&response=3Djson
>>>=20
>>> Sorted (by field), lower-case, encoded Command String:
>>>=20
>>> =
apikey=3D8v_gevjjgdjbbhibmlle4yyhkseqhrefztnv4up2fu3k9y12th7lscsn6-7seago1=
y
>>> c
>>> ctnute1ot0v7npcus8q&command=3Dlistvirtualmachines&response=3Djson
>>>=20
>>> SHA1 HEX String:
>>> f8d4d96dd59c3bd562dc32586539fa9162c5ed70
>>>=20
>>> SHA1 Base64 Signature (using SHA1 HEX String):
>>> 3wOrhy/SstxN+NbdoT8h/bkla2E
>>>=20
>>> Encoded Signature:
>>> 3wOrhy%2FSstxN%2BNbdoT8h%2Fbkla2E
>>>=20
>>> Final URL:
>>>=20
>>> =
http://10.217.5.192:8080/client/api?command=3DlistVirtualMachines&apiKey=3D=
8v
>>> _
>>> =
GEvJJgDjbbHIBmlle4yyHKseQhRefztnv4UP2fU3K9y12TH7lscsn6-7SEaGO1yCCTnUtE1oT
>>> 0
>>> v7npCuS8Q&response=3Djson&signature=3D3wOrhy%2FSstxN%2BNbdoT8h%2Fbkla2=
E
>>>=20
>>> Does the syntax of the final url, at least, look right?
>>>=20
>>> If so, then there must be something wrong with the signature.
>>>=20
>>> Regards,
>>> Evan
>>=20
>>=20
>>=20
>> The syntax looks right to me.
>>=20
>> -Alena.
>>=20
>>>=20
>>> - End -
>>>=20
>>>=20
>>> -Alena.
>>>=20
>>>>=20
>>>> SHA1 HEX String:
>>>> 9066d795102c0cf8a12322507887122b6b4a6095
>>>>=20
>>>> SHA1 Base64 Signature (using SHA1 HEX String and Secret Key):
>>>> SvDq03i4Tql9qkXuZwUDi3HfbH4
>>>>=20
>>>> Final URL:
>>>>=20
>>>> =
http://10.217.5.192:8080/client/api?command=3DlistVirtualMachines&apiKey=3D=
8
>>>> v
>>>> _
>>>> =
GEvJJgDjbbHIBmlle4yyHKseQhRefztnv4UP2fU3K9y12TH7lscsn6-7SEaGO1yCCTnUtE1o
>>>> T
>>>> 0
>>>> v7npCuS8Q&response=3Djson&signature=3DSvDq03i4Tql9qkXuZwUDi3HfbH4
>>>>=20
>>>> I am generating the SHA1 HEX String like so:
>>>> $digest =3D sha1_hex ($encode, $secret_key);
>>>> using this module:
>>>> use Digest::SHA qw(sha1 sha1_hex sha1_base64);
>>>>=20
>>>> I am generating the SHA1 Base64 Signature like so:
>>>> $signature =3D sha1_base64 ($digest, $secret_key);
>>>>=20
>>>>=20
>>>> Curious - Is SHA1 HEX different from HMAC SHA1?
>>>>=20
>>>>=20
>>>> Regards,
>>>> Evan
>>>>=20
>>>> -----Original Message-----
>>>> From: Prasanna Santhanam [mailto:prasanna.santhanam@citrix.com]
>>>> Sent: Friday, June 08, 2012 1:20 AM
>>>> To: cloudstack-users@incubator.apache.org
>>>> Subject: Re: Having trouble getting a CloudStack API URL with =
api_key
>>>> and
>>>> generated signature to work from the browser (Firefox).
>>>>=20
>>>> On Thu, Jun 07, 2012 at 10:38:20PM -0400, Evan Miller wrote:
>>>>> Hi:
>>>>>=20
>>>>> Right now, I am just testing a simple, final API-based url with
>>>>> signature in a browser.
>>>>> I am getting the following authentication related error in the
>>>>> browser:
>>>>>=20
>>>>> { "listvirtualmachinesresponse" : {"errorcode" : 401, "errortext" =
:
>>>>> "unable to verify user credentials and/or request signature"}  }
>>>>>=20
>>>>> Here is the final API-based url:
>>>>>=20
>>>>>=20
>>>>> =
http://10.217.5.192:8080/client/api?command=3DlistVirtualMachines&apiKey
>>>>> =
=3D8v_GEvJJgDjbbHIBmlle4yyHKseQhRefztnv4UP2fU3K9y12TH7lscsn6-7SEaGO1yCCT
>>>>> =
nUtE1oT0v7npCuS8Q&response=3Djson&signature=3D1ca7bc1bbc67b8f578c7d094c523=

>>>>> 537571ee17b1
>>>>>=20
>>>>> Here is how I built that final url:
>>>>>=20
>>>>> Using perl, it is based on the following pieces:
>>>>>=20
>>>>> my $cs_ip =3D '10.217.5.192';
>>>>> my $base_url =3D 'http://' . "$cs_ip" . ':8080'; my $api_path =3D
>>>>> '/client/api?'; my $api_key =3D
>>>>> =
'8v_gevjjgdjbbhibmlle4yyhkseqhrefztnv4up2fu3k9y12th7lscsn6-7seago1ycct
>>>>> nute1ot0v7npcus8q'; my $secret_key =3D
>>>>> =
'kNd2VxlXxCXwyJGlidr0ZcmcqXSH2refwxZTStD6If4vJu4QmJPOIui0rgr88mDI6DuGQ
>>>>> TzP9eQNOjlZBTReKg';
>>>>>=20
>>>>> The api_key and secret key come from my account (evan) in domain =
1.
>>>>> The keys were just generated in a new CloudStack GUI session.
>>>>> The evan account has ROOT Domain privileges.
>>>>>=20
>>>>> Then, following directions in the Developer's Guide ...
>>>>>=20
>>>>> The sorted, lower-case command string ($sorted_lc_cmd) is:
>>>>>=20
>>>>> =
apikey=3D8v_gevjjgdjbbhibmlle4yyhkseqhrefztnv4up2fu3k9y12th7lscsn6-7seag
>>>>> o1ycctnute1ot0v7npcus8q&command=3Dlistvirtualmachines&response=3Djso=
n
>>>>>=20
>>>>>=20
>>>>> I obtained the hex signature in perl as follows:
>>>>>=20
>>>>>=20
>>>>> $signature =3D hmac_sha1_hex ($sorted_lc_cmd, $secret_key);
>>>>>=20
>>>>>=20
>>>>> And, then, put together the final url, as above, with the =
calculated
>>>>> signature.
>>>>>=20
>>>>>=20
>>>>> What am I doing wrong or missing?
>>>>=20
>>>> Before you obtain the hmac SHA-1 you need to url encode the request =
url
>>>> stripping it off reserved characters like "+, !, $" etc. Then you
>>>> perform
>>>> the HMAC on the resultant string. This then is passed through a =
base64
>>>> encoder to obtain the signature. It looks like you missed this =
step. Can
>>>> you base64 encode the hmac result and give it a shot?
>>>>=20
>>>>=20
>>>>=20
>>>> --
>>>> Prasanna.,
>>>>=20
>>>>=20
>>>=20
>>>=20
>>>=20
>>=20
>>=20
>=20