deltacloud-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michal Fojtik <mfoj...@redhat.com>
Subject Fun with Bundler vs rubygems - ideas
Date Tue, 26 Mar 2013 14:18:03 GMT
Hi everyone,

First. Sorry for this long email :-)

I started working on refactoring of how we handle gems loading in 
Deltacloud API. This is what I discover:

1. In 'server', we don't use Bundler. If you run Deltacloud using:

'./bin/deltacloudd'

we use the old-school 'require' all over the DC server code (do a "grep 
'require ' -r lib/"). Having this, we always use the version of the gem 
which is currently available in the system. In most cases is the latest 
version available.

2. For 'server' tests, we do use Bundler (we call the Bundler.require in 
the Rakefile). So before you run 'rake test' you need to run 'bundle 
install'. This is not true, when the user want to execute just one 
single test file (ruby tests/.../test.rb) (yeah, he can use 'bundle ruby 
...', but srsly, how often you forget to do it ;-). In that case, we 
again trust the rubygems and use the system versions.

I think what we currently have is a bit mess :-) I found that up to 50% 
of problems, users have with DC installation is handling all various gem 
dependencies and versions (tilt, sinatra, rabbit, openstack, etc...). 
Also many bugs/jiras are related to breakages caused by incompatible 
changes in upstream gems we use.

So. How we can do things better? :-)

*** Idea ***:

Use bundler everywhere. I don't like bundler. But I think currently it 
is the only sane way how to manage gems in DC. We should check in the 
Gemfile.lock into 'server' dir, create an initialization file with 
Bundler.require(:default) and use this to require all gems we do need 
for server to operate. Also might we need to remove all "require 
'upstream_gem'" in DC, we we load the gems just once (when starting DC).

*** Problems ***:

- Different drivers require different gems. Having them all loaded by 
Bundler will increase DC boot time badly and increase the memory DC consume.

Solution:

We can create a 'group' for each driver in Bundler, like:

group :ec2 do
   gem 'aws'
end

group :rhevm do
   gem 'rbovirt', :version => '...'
end

The the 'driver' method in Deltacloud will need to be a bit smarter and 
call the "Bundler.require(driver_symbol)" when the driver is changed.

- We need to make sure that 'gem install deltacloud-core' work properly. 
Ruby gems does not play well with Gemfile.lock.

Solution:

We don't need 'gemspec', right? We do need gemspec **only** when we are 
building a gem (release). Can we generate 'gemspec' from what we have in 
Gemfile.lock before release? In that case we will make sure all gems we 
require have the correct versions (the one we tested on).

-------

I think having all gems managed by Bundler and loaded on one place 
(initializer) will make managing gems we use easier, where we can track 
what exact version we are currently supporting and we can avoid the 
issues similar to 'tilt' or 'sinatra'.

There is a good article about clarification how to use gemspec vs Gemfile:

http://yehudakatz.com/2010/12/16/clarifying-the-roles-of-the-gemspec-and-gemfile/

"When developing a gem, do not check your Gemfile.lock into version control"

vs.

"When developing an app, check in your Gemfile.lock"

In our case, we develop an app, right? The 'gem' we produce is just a 
distribution of this app.


What you think? Do you have a better idea?

   -- Michal

-- 

Michal Fojtik <mfojtik@redhat.com>
Deltacloud API, CloudForms

Mime
View raw message