deltacloud-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Koper, Dies" <di...@fast.au.fujitsu.com>
Subject RE: api test failures - how to resolve?
Date Fri, 25 Jan 2013 11:26:43 GMT
Hi Marios,

> thanks - can you please check if that one applies for you? I couldn't
> get it to apply - may need a rebase (or otherwise if it works for you
> let me know I can try again...)

I couldn't do a git pull with it so must have forgotten to do a pull
before it, sorry about that.
New one sent 5 min ago.

> discover stuff/skip accordingly. So with this in mind - for this kind
of
> 'fgcp specific' scenario, it may be better to look at adding to the
> white-box tests under /deltacloud/server/tests/drivers... i can't see
> fgcp tests there yet.

Ah you have a whitelist? That's sounds like a nice thing to add to the
readme.
Or better to keep it hidden to get people like me to try harder to get
them to pass? ;)

> gives a clue here. We are talking about creating an image from a
running
> server. In FGCP case, this is not possible. For starters, we shouldn't
> be advertising the 'create_image' action for running instances, when
you
> can't actually create the image from a running instance. If that logic
> were working correctly, this test would just be skipped (i.e.

Of course! The word snapshot confused me (I'm only used to snapshots of
volumes and images of instances), but the key here of course is
"running", and the tests don't start the instance, nor do they check its
state.
I'll look at adding code to check its state.

> fgcp instance running... no create_image...):

Yes, that's covered. The issue is with the test expecting that the
instance is running after create.
I'll look at starting it. In which case my waiting logic in the other
patch makes more sense too :)

> right - either what you suggest here - which is what other tests...
like
> 'instances_test' for example do with the 'setup' stage of creating the
> given resource before running the tests... or rethink the test itself
to
> be more lenient. Either way - its something that should be changed in
> the test - i.e. not in the fgcp driver.

In this case the test was part of code shared by all tests, so making it
more lenient (basically skipping all checks on elements, href etc.)
would affect a lot.
I'll look at doing what instances_test does.

Cheers,
Dies Koper


> -----Original Message-----
> From: marios@redhat.com [mailto:mandreou@redhat.com]
> Sent: Friday, 25 January 2013 7:58 PM
> To: dev@deltacloud.apache.org
> Cc: Koper, Dies
> Subject: Re: api test failures - how to resolve?
> 
> On 25/01/13 09:33, Koper, Dies wrote:
> > Hi Marios,
> >
> > I've been working on getting the api tests to run on the fgcp.
> > It's been a good experience as I found some bugs/room for
improvement
> > for the driver.
> >
> > I have just sent a patch to the ML with changes to both fgcp driver
and
> > tests that address most issues.
> > Please review.
> 
> thanks - can you please check if that one applies for you? I couldn't
> get it to apply - may need a rebase (or otherwise if it works for you
> let me know I can try again...)
> 
> >
> > May I have your feedback on the following:
> >
> > 1. With the fgcp, an instance needs to be started at least once (and
> > then stopped again) or else you can't create an image of it. So this
> > test fails. Do you have any suggestion how I can address it? There
is no
> > API to tell whether an instance has been started once.
> 
> ... can't think of an 'obvious' solution here. You could add some
> waiting logic (as you suggest for 3 below) but I agree with you that
> this may be unacceptable for other drivers; these tests are meant to
me
> 'driver agnostic' - i.e. you just point them at some running
deltacloud
> server with little/no knowledge of what's running there and they
> discover stuff/skip accordingly. So with this in mind - for this kind
of
> 'fgcp specific' scenario, it may be better to look at adding to the
> white-box tests under /deltacloud/server/tests/drivers... i can't see
> fgcp tests there yet.
> 
> I think this may be indicative of a problem with how we advertise
> actions. The name of the test:
> 
> 239 "it 'should allow to snapshot running instance if supported by
> provider' do"
> 
> gives a clue here. We are talking about creating an image from a
running
> server. In FGCP case, this is not possible. For starters, we shouldn't
> be advertising the 'create_image' action for running instances, when
you
> can't actually create the image from a running instance. If that logic
> were working correctly, this test would just be skipped (i.e. fgcp
> instance running... no create_image...):
> 
> 243 skip "no create image support for instance #{@@my_instance_id}"
> unless instance_actions.include?("create_image")$
> 
> This needs more thought - perhaps a JIRA would be a good start.
> 
> >
> > 2. Storage snapshots assume there are snapshots present. I get an
error
> > saying storage_snapshots must have some storage_snapshot elements
> > inside. I suppose the only way to address this one is to create a
volume
> > and then create a snapshot.
> >
> 
> right - either what you suggest here - which is what other tests...
like
> 'instances_test' for example do with the 'setup' stage of creating the
> given resource before running the tests... or rethink the test itself
to
> be more lenient. Either way - its something that should be changed in
> the test - i.e. not in the fgcp driver.
> 
> 
> > 3. Fgcp won't let me create an instance while another resource is
being
> > created. I need to add some waiting logic. I have a patch for that,
> > which works for fgcp, but I'm not sure it adds inacceptable waiting
time
> > for other drivers. I'll send it separately, as I don't want a NACK
for
> > this one delay an ACK for the first one.
> 
> well... ok let's deal with that one when we have the patch to point at
> and discuss,
> 
> 
> thanks, marios
> 
> >
> > Cheers,
> > Dies Koper
> >
> >
> >> -----Original Message-----
> >> From: Koper, Dies [mailto:diesk@fast.au.fujitsu.com]
> >> Sent: Friday, 30 November 2012 1:12 AM
> >> To: dev@deltacloud.apache.org
> >> Subject: api test failures - how to resolve?
> >>
> >> Hi,
> >>
> >> I've run the API tests on fgcp, i.e. rake test:deltacloud:
> >>
> >> 166 tests, 1340 assertions, 2 failures, 8 errors, 43 skips
> >>
> >> I'd like your suggestions on how to resolve the failures. My goal
is
> > to
> >> make changes so that a test run like this shows no failures for
fgcp
> > so
> >> that it can be used for easy regression testing.
> >>
> >>   1) Failure:
> >> test_0001_must have the "state" element defined for each instance
in
> >> collection(Deltacloud API instances collection)
> >> [d:/sources/OSS/cloud/deltacloud/tests/delt
> >> acloud/instances_test.rb:90]:
> >> Expected [] to not be empty.
> >>
> >> Fgcp driver requires to make an additional backend call for each
> >> instance in the collection, which caused http timeouts to occur
with
> > my
> >> (big) contract.
> >> Therefore, the fgcp driver only includes state when a single
instance
> > is
> >> retrieved, not for the whole collection.
> >> I thought that was acceptable, but here it says 'must'?
> >>
> >>   2) Error:
> >> test_0012_should allow to snapshot running instance if supported by
> >> provider(Deltacloud API instances collection):
> >> RestClient::ResourceNotFound: 404 Resource Not Found
> >>
> >> Fgcp only allows snapshots for stopped instances.
> >>
> >> I'll investigate the remaining ones tomorrow.
> >>
> >> Regards,
> >> Dies Koper
> >>
> >>
> >
> >
> 



Mime
View raw message