Mailing-List: contact dev-help@deltacloud.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@deltacloud.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Message-ID: <4FED6901.4040000@twgrid.org>
Date: Fri, 29 Jun 2012 16:36:17 +0800
From: Felix Lee <felix@twgrid.org>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:13.0) Gecko/20120615 Thunderbird/13.0.1
MIME-Version: 1.0
To: dev@deltacloud.apache.org
CC: "Ruben S. Montero" <rsmontero@opennebula.org>
Subject: Re: Poor performance with open nebula driver
References: <4FE7DB26.4040600@twgrid.org>
 <CAPvywewPtd6vejA90TQTF15fM=mjEPZqVry3KpXxWz=zoFBXcw@mail.gmail.com>
In-Reply-To: 
 <CAPvywewPtd6vejA90TQTF15fM=mjEPZqVry3KpXxWz=zoFBXcw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit

Hi Daniel,
Many thanks!!
I will test openebula_drive patch and give you feedback soon.
BTW, our production OpenNebula instance is in 3.2 now, so, maybe the 
occ_client patch will have no much effect for our instance....


Best regards,
Felix Lee ~

On 06/29/2012 04:19 PM, Daniel Molina wrote:
> Hi Felix
>
> On 25 June 2012 05:29, Felix Lee <felix@twgrid.org> wrote:
>> Dear All,
>> I am new with deltacloud, for integration requirement, we are evaluating
>> deltacloud solution.
>> Here, I don't know if anybody ever reported this issue, however, recently, I
>> noticed deltacloud is in poor performance with our open nebula instance.
>> Usually, our opennebula always runs over 500 VM instances, and while we use
>> deltacloud to query instances, it will take pretty long time to get all
>> results.
>>
>>
>> Here are some examples:
>>
>> With "deltacloudc instances" command:
>>
>> real    20m19.293s
>> user    0m7.842s
>> sys     0m0.681s
>>
>>
>> With curl to deltacloud rest api "/api/instances":
>> real    4m35.891s
>> user    0m0.002s
>> sys     0m0.005s
>>
>> With curl to opennebula OCCI interface "/compute" directly via the same
>> host:
>> real    0m1.249s
>> user    0m0.001s
>> sys     0m0.006s
>>
>
> In OpenNebula 3.2 when you retrieve the pool of instances through OCCI
> no extended information is included, hence you have to do one more
> request per instance to retrieve this information. In OpenNebula 3.4 a
> verbose option was included for OCCI pools. You can use GET
> /compute?verbose=true instead of GET /compute and the pool will
> contain extended information for each element.
>
> I have included a diff with the changes that would be required to
> support this param in the deltacloud driver, but I didn't test it.
> This should fix your performance issues.
>
> Also using Nokogiri instead of REXML should improve the driver performance.
>
> Hope this helps
>
>
> diff --git a/server/lib/deltacloud/drivers/opennebula/occi_client.rb
> b/server/lib/deltacloud/drivers/op
> index 8688995..7d71962 100644
> --- a/server/lib/deltacloud/drivers/opennebula/occi_client.rb
> +++ b/server/lib/deltacloud/drivers/opennebula/occi_client.rb
> @@ -83,8 +83,8 @@ module OCCIClient
>       ######################################################################
>       # Retieves the pool of Virtual Machines
>       ######################################################################
> -    def get_vms
> -      get('/compute')
> +    def get_vms(verbose=false)
> +      get('/compute', verbose)
>       end
>
>       ######################################################################
> @@ -196,8 +196,8 @@ module OCCIClient
>       ######################################################################
>       # Retieves the pool of Images owned by the user
>       ######################################################################
> -    def get_images
> -      get('/storage')
> +    def get_images(verbose=false)
> +      get('/storage', verbose)
>       end
>
>
> @@ -275,10 +275,16 @@ module OCCIClient
>
>       private
>
> -    def get(path)
> +    def get(path, verbose=false)
>         url = URI.parse(@endpoint+path)
> +
> +      params = []
> +      params << "verbose=true" if verbose
> +      params << "#{url.query}" if url.query
> +
>         path = url.path
> -      path << "?#{url.query}" if url.query
> +      path << "?#{params.join('&')}"
> +
>         req = Net::HTTP::Get.new(path)
>
>         do_request(url, req)
> diff --git a/server/lib/deltacloud/drivers/opennebula/opennebula_driver.rb
> b/server/lib/deltacloud/driv
> index cc1f13e..7d2385f 100644
> --- a/server/lib/deltacloud/drivers/opennebula/opennebula_driver.rb
> +++ b/server/lib/deltacloud/drivers/opennebula/opennebula_driver.rb
> @@ -84,13 +84,11 @@ class OpennebulaDriver < Deltacloud::BaseDriver
>     def images(credentials, opts=nil)
>       occi_client = new_client(credentials)
>
> -    xml = treat_response(occi_client.get_images)
> +    xml = treat_response(occi_client.get_images(true))
>
>       # TBD Add extended info in the pool
>       images = REXML::Document.new(xml).root.elements.map do |d|
> -      im_id = d.attributes['href'].split("/").last
> -      storage = treat_response(occi_client.get_image(im_id))
> -      convert_image(storage, credentials)
> +      convert_image(d, credentials)
>       end
>     end
>
> @@ -156,13 +154,11 @@ class OpennebulaDriver < Deltacloud::BaseDriver
>     def instances(credentials, opts=nil)
>       occi_client = new_client(credentials)
>
> -    xml = treat_response(occi_client.get_vms)
> +    xml = treat_response(occi_client.get_vms(true))
>
>       # TBD Add extended info in the pool
>       instances = REXML::Document.new(xml).root.elements.map do |d|
> -      vm_id = d.attributes['href'].split("/").last
> -      computexml = treat_response(occi_client.get_vm(vm_id))
> -      convert_instance(computexml, credentials)
> +      convert_instance(d, credentials)
>       end
>
>       instances = filter_on( instances, :state, opts )
>

-- 
Felix Lee                               Academia Sinica Grid & Cloud.
Tel: +886-2-27898308
Office: Room P111, Institute of Physics, 128 Academia Road, Section 2, 
Nankang, Taipei 115, Taiwan